Merge branch 'release/0.23' into feature/1488-success-rate

CHANGELOG.md

@@ -1,5 +1,31 @@
 # InSTEDD Surveda Changelog
 
+## 0.23
+
+### Features
+
+* Survey folders [#1463](https://github.com/instedd/surveda/issues/1463)
+* Prometheus metrics [#1472](https://github.com/instedd/surveda/issues/1472), [#1473](https://github.com/instedd/surveda/issues/1473)
+* Prefix respondent IDs with `r` [#1497](https://github.com/instedd/surveda/issues/1497)
+* Success rate and queue size cockpit [#1488](https://github.com/instedd/surveda/issues/1488)
+
+### Bugfixes
+
+* 'Quota completed steps' section cannot be disabled [#1490](https://github.com/instedd/surveda/issues/1490)
+* Cannot change the sections order [#1496](https://github.com/instedd/surveda/issues/1496)
+* Call duration is not computed on failed calls [#1502](https://github.com/instedd/surveda/issues/1502)
+
+## Viburnum 0.22.1
+
+### Features
+
+* Add a column to the respondents CSV with the order of the sections [#1492](https://github.com/instedd/surveda/issues/1492)
+* Enable remote shell for Elixir process [#1511](https://github.com/instedd/surveda/issues/1511)
+
+### Bugfixes
+
+* Dropdown does not open on first click [#1491](https://github.com/instedd/surveda/issues/1491)
+
 ## Viburnum 0.22
 
 ### Features

Dockerfile

@@ -23,4 +23,4 @@ RUN mix phx.digest
 ENV PORT=80
 EXPOSE 80
 
-CMD mix phx.server
+CMD elixir --sname server -S mix phx.server

README.md

@@ -1,4 +1,4 @@
-# Ask
+# Surveda
 
 ## Dockerized development
 
@@ -8,16 +8,18 @@ To run the app: `docker-compose up`
 
 Now you can visit [`localhost:4000`](http://localhost:4000) from your browser.
 
-To open a shell in a container: `docker exec -it ask_db_1 bash`, where `ask_db_1` is a container name. You can list containers with `docker ps`.
+To open a shell in a container: `docker-compose exec db bash`, where `db` is the service name. You can list the services `docker-compose ps --services`.
 
-To start an Elixir console in your running Phoenix app container: `docker exec -it ask_app_1 iex -S mix`.
+To start an Elixir console in your running Phoenix app container: `docker-compose exec app iex -S mix`.
 
-## Exposing your containers as *.ask.dev
+## Exposing your containers as *.surveda.lvh.me
 
-You can use [dockerdev](https://github.com/waj/dockerdev) to access the web app at `app.ask.dev` and ngrok at `ngrok.ask.dev`.
+You can use [dockerdev](https://github.com/waj/dockerdev) to access the web app at `app.surveda.lvh.me` and ngrok at `ngrok.surveda.lvh.me`.
 
 Just follow the instructions at the README of dockerdev.
 
+**WARNING:** You should install `dockerdev` _before_ creating your stack's network in Docker. If you have already run `./dev-setup.sh`, you may want to run `docker-compose down -v` to **delete every container, data and other artifacts** from the project and start from scratch _after_ running `dockerdev`.
+
 ## Linting and Formatting
 
 To help us keep a consistent coding style, we're using StandardJS. Follow their instructions to install it: http://standardjs.com/#install
@@ -55,6 +57,50 @@ To setup a channel you need to create it using the console. For that you need to
 
 In order for it to work, that Verboice channel must be associated to a dummy flow of a dummy Verboice project. Otherwise it will fail and won't log anything.
 
+## GUISSO
+
+You need GUISSO to access Verboice and/or Nuntium channels.
+
+Get a working GUISSO instance (online, or hosted on your development machine) and create a new Application. If it's a local Guisso instance, use `app.surveda.lvh.me` as the domain, and this two redirect URIs:
+
+```
+http://app.surveda.lvh.me/session/oauth_callback
+http://app.surveda.lvh.me/oauth_client/callback
+```
+
+To work with a cloud GUISSO, make sure your `ngrok` service is running (`docker-compose up ngrok`), and get your ngrok domain visiting `http://ngrok.surveda.lvh.me`. Fill the Application information as for the local case, but using the ngrok domain instead. When you restart your `ngrok` service, you will need to update this information before approving new authorizations in GUISSO.
+
+On your local surveda directory, create a `config/local.exs` file like below, including the client ID & secret from your Application in GUISSO:
+
+```
+use Mix.Config
+
+config :alto_guisso,
+  enabled: true,
+  base_url: "http://web.guisso.lvh.me", # or https://login-stg.instedd.org for a cloud GUISSO
+  client_id: "<your app's client id in guisso>",
+  client_secret: "<your app's client secret in guisso>",
+
+config :ask, Ask.Endpoint,
+  url: [host: "app.surveda.lvh.me"] # or "abcd123.ngrok.io" for a cloud GUISSO
+```
+
+### Verboice Channel
+
+Once you have GUISSO enabled on Surveda, you can connect a Verboice instance that's already registered with GUISSO by adding this fragment to your `config/local.exs`:
+
+```
+config :ask, Verboice,
+  base_url: "http://web.verboice.lvh.me", # or the URL for your Verboice instance
+  channel_ui: true,
+  guisso: [
+    base_url: "http://web.guisso.lvh.me", # or the URL for your GUISSO
+    client_id: "<Surveda's client id>",
+    client_secret: "<Surveda's client secret>",
+    app_id: "web.verboice.lvh.me" # or your Verboice APP ID in GUISSO
+  ]
+```
+
 ## Coherence
 
 ### Upgrading
@@ -77,4 +123,4 @@ Surveda will forward any conversation with a logged user identifying them throug
 
 If you don't want to use Intercom, you can simply omit `INTERCOM_APP_ID` or set it to `''`.
 
-To test the feature in development, add the `INTERCOM_APP_ID` variable and its value to the `environment` object inside the `app` service in `docker-compose.yml`.
+To test the feature in development, add the `INTERCOM_APP_ID` variable and its value to the `environment` object inside the `app` service in `docker-compose.yml`.

config/config.exs

@@ -85,14 +85,10 @@ config :coherence, Ask.Coherence.Mailer,
 # %% End Coherence Configuration %%
 
 config :prometheus, Ask.PrometheusExporter,
-  auth: {:basic,
-    quote do
-      System.get_env("METRICS_USER") || "metrics"
-    end,
-    quote do
-      System.get_env("METRICS_PASS")
-    end
-  }
+  auth: false
+
+config :ask, Ask.MetricsEndpoint,
+  http: [port: 9980]
 
 config :alto_guisso,
   enabled: System.get_env("GUISSO_ENABLED") == "true",

docker-compose.yml

@@ -30,8 +30,10 @@ services:
       DATABASE_HOST: db
       HTTP_PORT: 80
       TERM: xterm
+      VIRTUAL_PORT: 80
     ports:
-      - 4000:80
+      - 80
+      - 9980:9980
 
   webpack:
     build:
@@ -50,6 +52,6 @@ services:
   ngrok:
     image: wernight/ngrok
     ports:
-      - 4040:4040
+      - 4040
     environment:
       NGROK_PORT: 'app:80'

lib/ask.ex

@@ -16,6 +16,7 @@ defmodule Ask do
       supervisor(Ask.Repo, []),
       # Start the endpoint when the application starts
       supervisor(Ask.Endpoint, []),
+      supervisor(Ask.MetricsEndpoint, [])
       # Start your own worker by calling: Ask.Worker.start_link(arg1, arg2, arg3)
       # worker(Ask.Worker, [arg1, arg2, arg3]),
     ]

lib/ask/ecto_types/stats.ex

@@ -2,38 +2,26 @@ defmodule Ask.Stats do
   @behaviour Ecto.Type
   alias __MODULE__
 
-  defstruct total_received_sms: 0, total_sent_sms: 0, total_call_time: 0, current_call_first_interaction_time: nil, current_call_last_interaction_time: nil
+  defstruct total_received_sms: 0, total_sent_sms: 0, total_call_time: 0
 
   def type, do: :longtext
 
   def cast(%Stats{} = stats) do
     {:ok, stats}
   end
 
-  def cast(%{current_call_first_interaction_time: time} = stats) when is_binary(time) do
-    {:ok, time, _} = time |> DateTime.from_iso8601
-    cast(%{stats | current_call_first_interaction_time: time})
-  end
-  def cast(%{current_call_last_interaction_time: time} = stats) when is_binary(time) do
-    {:ok, time, _} = time |> DateTime.from_iso8601
-    cast(%{stats | current_call_last_interaction_time: time})
-  end
-  def cast(%{total_received_sms: total_received_sms, total_sent_sms: total_sent_sms, total_call_time: total_call_time, current_call_first_interaction_time: current_call_first_interaction_time, current_call_last_interaction_time: current_call_last_interaction_time}) do
+  def cast(%{total_received_sms: total_received_sms, total_sent_sms: total_sent_sms, total_call_time: total_call_time}) do
     {:ok, %Stats{
       total_received_sms: total_received_sms,
       total_sent_sms: total_sent_sms,
-      total_call_time: total_call_time,
-      current_call_first_interaction_time: current_call_first_interaction_time,
-      current_call_last_interaction_time: current_call_last_interaction_time
+      total_call_time: total_call_time
     }}
   end
   def cast(%{} = map) do
     cast(%{
       total_received_sms: map["total_received_sms"],
       total_sent_sms: map["total_sent_sms"],
-      total_call_time: map["total_call_time"],
-      current_call_first_interaction_time: map["current_call_first_interaction_time"],
-      current_call_last_interaction_time: map["current_call_last_interaction_time"]
+      total_call_time: map["total_call_time"]
     })
   end
   def cast(nil), do: {:ok, %Stats{}}
@@ -60,37 +48,11 @@ defmodule Ask.Stats do
     %{stats | total_call_time: count}
   end
 
-  def first_interaction_time(%Stats{current_call_first_interaction_time: date_time}), do: date_time
-  def first_interaction_time(%Stats{} = stats, date_time) do
-    %{stats | current_call_first_interaction_time: date_time}
-  end
-
-  def last_interaction_time(%Stats{current_call_last_interaction_time: date_time}), do: date_time
-  def last_interaction_time(%Stats{} = stats, date_time) do
-    %{stats | current_call_last_interaction_time: date_time}
-  end
-
   def add_received_sms(%Stats{total_received_sms: total} = stats, count \\ 1) do
     %{stats | total_received_sms: total + count}
   end
 
   def add_sent_sms(%Stats{total_sent_sms: total} = stats, count \\ 1) do
     %{stats | total_sent_sms: total + count}
   end
-
-  def set_interaction_time(%Stats{current_call_first_interaction_time: nil} = stats, date_time) do
-    first_interaction_time(stats, date_time)
-  end
-  def set_interaction_time(%Stats{} = stats, date_time) do
-    last_interaction_time(stats, date_time)
-  end
-
-  def add_total_call_time(%Stats{current_call_first_interaction_time: nil} = stats), do: stats
-  def add_total_call_time(%Stats{current_call_last_interaction_time: nil} = stats), do: stats
-  def add_total_call_time(%Stats{current_call_first_interaction_time: first_interaction_time, current_call_last_interaction_time: last_interaction_time} = stats) do
-    stats
-    |> total_call_time(DateTime.diff(last_interaction_time, first_interaction_time) / 60)
-    |> first_interaction_time(nil)
-    |> last_interaction_time(nil)
-  end
 end

lib/ask/endpoint.ex

@@ -34,6 +34,5 @@ defmodule Ask.Endpoint do
     key: "_ask_key",
     signing_salt: "/wP7KV3P"
 
-  plug Ask.PrometheusExporter
   plug Ask.Router
 end

lib/ask/runtime/session.ex

@@ -16,14 +16,22 @@ defmodule Ask.Runtime.Session do
       current_mode: SessionModeProvider.new(mode, channel, retries),
       fallback_mode: SessionModeProvider.new(fallback_mode, fallback_channel, fallback_retries),
       flow: flow,
-      respondent: respondent,
+      respondent: update_section_order(respondent, flow.section_order),
       fallback_delay: fallback_delay,
       count_partial_results: count_partial_results,
       schedule: schedule
     }
     run_flow(session)
   end
 
+  def update_section_order(respondent, nil), do: respondent
+
+  def update_section_order(respondent, section_order) do
+    respondent
+    |> Respondent.changeset(%{section_order: section_order})
+    |> Repo.update!
+  end
+
   def default_fallback_delay do
     @default_fallback_delay
   end

lib/ask/runtime/verboice_channel.ex

@@ -1,7 +1,7 @@
 defmodule Ask.Runtime.VerboiceChannel do
   alias __MODULE__
   use Ask.Web, :model
-  alias Ask.{Repo, Respondent, Channel, Stats, SurvedaMetrics}
+  alias Ask.{Repo, Respondent, Channel, SurvedaMetrics, Stats}
   alias Ask.Runtime.{Broker, Flow, Reply}
   alias Ask.Router.Helpers
   import Plug.Conn
@@ -232,32 +232,19 @@ defmodule Ask.Runtime.VerboiceChannel do
     Broker.channel_failed(respondent, status)
   end
 
-  def update_interactions(respondent) do
-    respondent = Respondent
-    |> Repo.get(respondent.id)
-
-    stats = respondent.stats
-    |> Stats.set_interaction_time(DateTime.utc_now())
-
-    respondent
-    |> Respondent.changeset(%{stats: stats})
-    |> Repo.update!
-  end
-
-  def update_call_time(respondent) do
-    respondent = Respondent
-    |> Repo.get(respondent.id)
-
+  def update_call_time(respondent, call_time) do
     stats = respondent.stats
-    |> Stats.add_total_call_time()
+    |> Stats.total_call_time(call_time)
 
     respondent
     |> Respondent.changeset(%{stats: stats})
     |> Repo.update!
   end
 
-  def callback(conn, %{"path" => ["status", respondent_id, _token], "CallStatus" => status} = params) do
+  def callback(conn, %{"path" => ["status", respondent_id, _token], "CallStatus" => status, "CallDuration" => call_duration_seconds} = params) do
+    call_duration = call_duration_seconds |> String.to_integer
     respondent = Repo.get!(Respondent, respondent_id)
+    |> update_call_time(call_duration / 60)
     case status do
       s when s in ["failed", "busy", "no-answer", "expired"] ->
         channel_failed(respondent, status, params)
@@ -285,19 +272,15 @@ defmodule Ask.Runtime.VerboiceChannel do
           digits -> Flow.Message.reply(digits)
         end
 
-        update_interactions(respondent)
-
         case broker.sync_step(respondent, response, "ivr") do
           {:reply, reply} ->
             prompts = Reply.prompts(reply)
             num_digits = Reply.num_digits(reply)
             gather(respondent, prompts, num_digits)
           {:end, {:reply, reply}} ->
             prompts = Reply.prompts(reply)
-            update_call_time(respondent)
             say_or_play(prompts) ++ [hangup()]
           :end ->
-            update_call_time(respondent)
             hangup()
         end
     end

priv/repo/migrations/20190821214420_add_section_order_to_respondent.exs

@@ -0,0 +1,9 @@
+defmodule Ask.Repo.Migrations.AddSectionOrderToRespondent do
+  use Ecto.Migration
+
+  def change do
+    alter table(:respondents) do
+      add :section_order, :string
+    end
+  end
+end