Use the built-in JSON library

[elfenlaid]

Address #6461

The changes set the built-in JSON library as a default JSON library.

The idea is to check whether the built-in JSON library is present and use it if so:

  @default_json_library if Code.ensure_loaded?(JSON), do: JSON, else: Jason

  def json_library do
    Application.get_env(:phoenix, :json_library, @default_json_library)
  end

Correct me if I'm wrong, but I think a naive check of whether the library is present during compilation time should work, as it's a built-in library.

---

What I'm not sure about is whether such a move results in a breaking change.

It could be a breaking change for projects that don't specify the json_library and use @derive Jason.Encoder on structures passed to Phoenix.Controller.json/2. That sounds pretty niche, especially given that the template project includes:

config :phoenix, :json_library, Jason

Also, I'm specifically leaving the jason dependency in place for pre-27 OTP versions.

---

Please let me know what you think and whether the changes make sense. In other words, maybe it would be better to stick with Jason until Phoenix requires OTP 28.

---

Similar discussions / changes:

Use Jason instead of Poison for json encoding by chrismccord #2734phoenixframework/phoenix
Investigate using Jason instead of Poison #2693phoenixframework/phoenix

Use JSON instead of Jason by kubosuke #234felt/geo

[elfenlaid]

Not the most elegant solution, but...

[SteffenDE]

Looks good to me! But if we actually change new projects to use JSON, we'll also need to bump the minimum version we set to Elixir 1.18, see

phoenix/installer/mix.exs

Lines 12 to 20 in 9fc21c6

	# If the elixir requirement is updated, we need to update:
	#
	# 1. all mix.exs generated by the installer
	# 2. guides/introduction/installation.md
	# 3. guides/deployment/releases.md
	# 4. test/test_helper.exs at the root
	# 5. installer/lib/mix/tasks/phx.new.ex
	#
	@elixir_requirement "~> 1.15"

I wonder if we should postpone this change to a future Phoenix 1.9? @josevalim

[josevalim]

Yes, v1.9 material for sure. But it looks good. We can probably assign a v1.9 milestone and merge it once we finally branch v1.8 out. I don’t think it is worth branching out though, as I dont think there is anything beyond this PR for now?

[elfenlaid]

Yes, v1.9 material for sure. But it looks good. We can probably assign a v1.9 milestone and merge it once we finally branch v1.8 out. I don’t think it is worth branching out though, as I dont think there is anything beyond this PR for now?

I don't think there's anything else I can add to this PR. Thank you everyone for looking into this! 🙇

[saleyn]

Looking at this PR - I believe it's not flexible enough to allow support of different JSON libraries beyond Jason, and :json. It would be better to define a behavior for encoding/decoding, and whatever library is used, to ensure that it supports that behavior. I am the maintainer of glazer, and would love to be able to use that JSON parsing library with Phoenix for performance reasons.

[josevalim]

I’d recommend modules to provide a similar interface to the JSON in Elixir, as that will become effectively standard over time. And most of it was mirrored after Jason anyway for compatibility.

[saleyn]

What would be a downside of providing a JSON behavior for phoenix?

[josevalim]

The issue is that Ecto has its own library, Ash as well, and so on. If each of them need a behavior, you would then need all of them to agree or you end up with several wrappers. And if they are going to agree on one, the Elixir interface will most likely be the one to choose.

[saleyn]

What I am asking is to document it in some form (could be at the Elixir level) so that library developers could follow the spec. Right now there's none, which opens the doors for undesirably having to deal with several JSON library dependencies in your project (because Ecto, Phoenix, Ash, Bandit, etc. have their own API), when all you want is to have one pluggable JSON library of your choice. Having a JSON behavior at the Elixir level could help, but even if it's just formally defined in documentation, we'd be at a better place.

[SteffenDE]

@saleyn then send a PR please that adds a sentence to https://phoenix.hexdocs.pm/Phoenix.html#json_library/0, e.g.

Phoenix expects this to be a module with decode!/1, decode_to_iodata!/1, encode!/1, and encode_to_iodata!/1 functions.

[saleyn]

@saleyn then send a PR please that adds a sentence to https://phoenix.hexdocs.pm/Phoenix.html#json_library/0, e.g.

Phoenix expects this to be a module with decode!/1, decode_to_iodata!/1, encode!/1, and encode_to_iodata!/1 functions.

I will submit a PR. @SteffenDE, I know you would probably hate this, but I still have to ask. Would it be possible to require that the canonical function names for the JSON library interface not end with '!', so that the Erlang libraries could be used without requiring them to have an Elixir wrapper?

[josevalim]

@saleyn if we change the API, then all existing uses of JSON and Jason then become broken, no?

[saleyn]

Phoenix expects this to be a module with decode!/1, decode_to_iodata!/1, encode!/1, and encode_to_iodata!/1 functions.

@saleyn if we change the API, then all existing uses of JSON and Jason then become broken, no?

This highlights a problem that was created when JSON's interface was designed. I guess the design didn't consider that it would become the standard for cross-languages. Maybe it can be extended with adding decode/1, decode_to_iodata/1, encode/1, and encode_to_iodata/1 that return {:ok|:error, result|reason} tuples, and that would become the JSON's API for component designers, whereas the former 4 functions would use them under the hood?

On the other hand, I was under the impression that Erlang used to not permit '!' in function names, but I just tried it, and it does seem to work:

$ cat mod.erl
-module(mod).
-export(['f!'/0]).

'f!'() ->
  os:system_time().
$ erlc mod.erl
$ erl
Erlang/OTP 29 [erts-17.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit:ns]

Eshell V17.0 (press Ctrl+G to abort, type help(). for help)
1> mod:'f!'().
1781527995879131559

So, my comment about the need to change the API is moot.

@josevalim are there any other concerns for an Erlang library to expose decode!/1, decode_to_iodata!/1, encode!/1, and encode_to_iodata!/1 for use in Elixir? If not, I'll add them to glazer.

[josevalim]

Yes, the design happened by accident, unfortunately. And yes, you can totally have ! in Erlang functions and that should work fine!!!

[saleyn]

Consider :persistent_term, which is roughly 30-100% faster:

defmodule JSON.Adapter do
  @default_json_library if Code.ensure_loaded?(JSON), do: JSON, else: Jason

  def json_library do
    case :persistent_term.get(:json_library, nil) do
      nil ->
        module = :application.get_env(:phoenix, :json_library, @default_json_library)
        :persistent_term.put(:json_library, module)
        module
      module ->
        module
    end
  end
end

[elfenlaid]

TIL persistent_term and it looks cool, but I'm not sure if we are in the position of adding anything significant to this PR. I'm happy to add it, but it's @SteffenDE call at this point

[rhcarvalho]

In the meantime HexDocs URLs now use subdomains.

The installer could use runtime detection to decide on the JSON library for new apps, that will keep the installer compatible with Elixir pre-1.18.

[rhcarvalho]

I think here we could do a similar runtime detection as in https://github.com/phoenixframework/phoenix/pull/6481/changes#r2356177940, and only configure JSON as the default for Elixir versions where its available.

Same for the umbrella app config template.

[elfenlaid]

It's a nice idea, yet I don't think it would fly.

The first option is to check Elixir version during phx.new, but the Elixir version that generates the project is not guaranteed to match the Elixir version that runs the project. So it sounds like a breaking change.

The second option is to embed this check into new projects. Technically it should work, but I'm not sure that such checks are awkward and probably are not something you would like to see in new projects.

[saleyn]

@saleyn then send a PR please that adds a sentence to https://phoenix.hexdocs.pm/Phoenix.html#json_library/0, e.g.

Phoenix expects this to be a module with decode!/1, decode_to_iodata!/1, encode!/1, and encode_to_iodata!/1 functions.

@SteffenDE, could you clarify if this was a typo - there's no JSON.decode_to_iodata! in JSON. Did you mean that the library would need to support only 3 functions: decode!/1, encode_to_iodata!/1, encode!/1?

[josevalim]

Did you mean that the library would need to support only 3 functions: decode!/1, encode_to_iodata!/1, encode!/1?

yes!

[saleyn]

Please check my PR above (#6732).

As a reference implementation, the glazer_json module (part of the glazer Erlang JSON library) exposes these three functions under their quoted Erlang names — 'decode!'/1, 'encode!'/1, and 'encode_to_iodata!'/1 — and can be configured directly:

config :phoenix, :json_library, :glazer_json

For parity with Elixir's JSON module, where null decodes to and encodes from nil rather than the atom :null, these three functions apply that mapping automatically.