-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Adrien Moreau
committed
Dec 1, 2021
1 parent
5e20f1c
commit 9e84826
Showing
8 changed files
with
151 additions
and
109 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -23,3 +23,6 @@ erl_crash.dump | |
http_client-*.tar | ||
|
||
.elixir_ls/ | ||
|
||
*.plt | ||
*.xml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,21 +1,31 @@ | ||
# HttpClient | ||
|
||
**TODO: Add description** | ||
Httpoison boosted with telemetry, mox and ex_rated. | ||
|
||
```elixir | ||
HttpClient.get("http://mydomain.com") | ||
``` | ||
|
||
## Installation | ||
|
||
If [available in Hex](https://hex.pm/docs/publish), the package can be installed | ||
by adding `http_client` to your list of dependencies in `mix.exs`: | ||
The package can be installed by adding `http_client` to your list of dependencies | ||
in `mix.exs`: | ||
|
||
```elixir | ||
def deps do | ||
[ | ||
{:http_client, "~> 0.1.0"} | ||
{:http_client, "~> 0.2.3"} | ||
] | ||
end | ||
``` | ||
|
||
Documentation can be generated with [ExDoc](https://github.com/elixir-lang/ex_doc) | ||
and published on [HexDocs](https://hexdocs.pm). Once published, the docs can | ||
be found at [https://hexdocs.pm/http_client](https://hexdocs.pm/http_client). | ||
The docs can be found at [https://hexdocs.pm/http_client](https://hexdocs.pm/http_client). | ||
|
||
## Configuration | ||
|
||
```elixir | ||
# Will rate limit calls to mydomain.com to 5 per seconds | ||
# HttpClient.get("http://mydomain.com/any_path") | ||
config :http_client, :rate_limits, "mydomain.com": [{:timer.seconds(1), 5}] | ||
``` | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
defmodule HttpClient.RateLimiter do | ||
@moduledoc """ | ||
Module used to rate limit calls to certain host | ||
In `config.exs` : | ||
config :http_client, :rate_limits, | ||
"google.com": [{:timer.seconds(1), 5}, {:timer.hours(24), 500_000}] | ||
""" | ||
|
||
@spec rate_limit(String.t()) :: :ok | ||
def rate_limit(url) do | ||
case retrieve_rate_limit_config(url) do | ||
{:ok, {host, rate_limits}} -> | ||
:ok = enforce_rate_limits(host, rate_limits) | ||
|
||
:error -> | ||
:ok | ||
end | ||
end | ||
|
||
@spec enforce_rate_limits(atom(), list({non_neg_integer(), non_neg_integer()})) :: :ok | ||
defp enforce_rate_limits(_host, []), do: :ok | ||
|
||
defp enforce_rate_limits(host, [{scale, limit} | remaining_limits] = rate_limits) do | ||
bucket = {host, scale} | ||
{_, _, ms_to_next_bucket, _, _} = ExRated.inspect_bucket(bucket, scale, limit) | ||
|
||
case ExRated.check_rate(bucket, scale, limit) do | ||
{:ok, _} -> | ||
enforce_rate_limits(host, remaining_limits) | ||
|
||
_ -> | ||
Process.sleep(ms_to_next_bucket) | ||
enforce_rate_limits(host, rate_limits) | ||
end | ||
end | ||
|
||
@spec retrieve_rate_limit_config(String.t()) :: | ||
{:ok, {atom(), list({non_neg_integer(), non_neg_integer()})}} | :error | ||
defp retrieve_rate_limit_config(url) do | ||
%URI{host: host} = URI.parse(url) | ||
host_atom = String.to_existing_atom(host) | ||
{:ok, rate_limits} = Application.fetch_env(:http_client, :rate_limits) | ||
{:ok, value} = Keyword.fetch(rate_limits, host_atom) | ||
{:ok, {host_atom, value}} | ||
rescue | ||
_ -> :error | ||
end | ||
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.