diff --git a/docs/build/api.md b/docs/build/api.md new file mode 100644 index 0000000..afedcd6 --- /dev/null +++ b/docs/build/api.md @@ -0,0 +1,6 @@ + +Many if not all of the functions detailed in this API are really just convenience functions wrapped around methods in the [`ODBC.jl`](https://github.com/JuliaDB/ODBC.jl) package, with some syntax specific to how we have set up our databases. + + +Often a `dsn` argument is requested. You can provide `ICDataServer.dsn`. + diff --git a/docs/build/config.md b/docs/build/config.md new file mode 100644 index 0000000..05d2fe9 --- /dev/null +++ b/docs/build/config.md @@ -0,0 +1,131 @@ + +Several configuration files should be generated and put into the `deps` directory of the ICDataServer package. These are not tracked by git as they may contain sensitive information and in any case are specific to the particular installation. These are used by [`ICDataServer.setuptables`](config.md#ICDataServer.setuptables) to add default information. + +# +**`ICDataServer.setuptables`** — *Function*. + + + +``` +setuptables() +``` + +Initialize a database with the tables needed to run ICDataServer: + + * `users` + * `servers` + * `instrumentkinds` + * `instruments` + * `jobs` + * `notes` + +The creation of the tables can proceed without additional input, but in some cases it may make sense for the tables to be automatically filled with data. + +If new instrument kinds are added to `deps/instrumentkinds.json`, they will be added to `instrumentkinds` table. If kinds are encountered in the database that are not in the file, they remain in the database and are not deleted. + +If new users are added to `deps/users.json`, they will be "upserted" into the `users` table (users are created if necessary; if an existing user name is attempted to be inserted into the table, then that user has their info updated from the json file). Existing users not found in users.json remain in the table. + + +source
+ + + + +### deps/config.json + + +This JSON file consists of up to four keys and largely exists to provide connection information between the database and InstrumentControl clients. Typical example: + + +``` +{ + "jobsock":"tcp://*:50001", + "dsn":"icdataserver", +} +``` + + +Additional fields not shown in the example may include: + + + * "username": A username to connect to the database, if not provided when the DSN was configured. + * "password": A password to connect to the database, if not provided when the DSN was configured. + + + + +### deps/instrumentkinds.json + + +This JSON file declares the various kinds of instruments. Its purpose is to standardize such declarations in the database. Typical example: + + +``` +{ + "kind": [ + "awg", + "dcsource", + "rfsource", + "vna" + ] +} +``` + + + + +### deps/servers.json + + +Declaration of servers. Typical example: + + +``` +{ + "servers": [ + { + "alias":"local_data", + "address":"127.0.0.1", + "port":"50002" + }, + { + "alias":"local_hw", + "address":"127.0.0.1", + "port":"50003" + } + ] +} +``` + + + + +### deps/users.json + + +Declaration of users, including contact information which may be used by the software to send alerts and notifications. Typical example: + + +``` +{ + "users": [ + { + "username":"default", + "first":"No", + "last":"Name" + }, { + "username":"anotheruser", + "first":"Really", + "middle":"No", + "last":"Name", + "email":"someaddress@caltech.edu", + "phone":"15555551212", + "office":"Watson 5th floor" + } + ] +} +``` + + +The only required fields for each dictionary are `username`, `first`, `last`. + diff --git a/docs/build/design.md b/docs/build/design.md index 5959f36..0d14d60 100644 --- a/docs/build/design.md +++ b/docs/build/design.md @@ -1,8 +1,3 @@ - - -## Design - - ICDataServer.jl diff --git a/docs/build/index.md b/docs/build/index.md index 437788b..b6b4184 100644 --- a/docs/build/index.md +++ b/docs/build/index.md @@ -12,9 +12,28 @@ Automatic logging of measurement data and metadata. ## Installation - * Install the most recent version of [PostgreSQL](https://www.postgresql.org/download/) - * Install the 64-bit ODBC driver for PostgreSQL (on Windows, it is an optional installation prompted automatically). - * `Pkg.clone("https://github.com/painterqubits/ICDataServer.jl.git")` +Estimated burden: twenty minutes. + + + * Install the most recent version of [PostgreSQL](https://www.postgresql.org/download/). When setting up, a password for the default user (`postgres`) will be requested. Choose something secure. + * Following the installation, Stack Builder will open. This lets you add components to the PostgreSQL installation. Install from Database Drivers (64-bit when there is an option): + + * ODBC driver (only one of the three that is currently utilized) + * JDBC driver + * Npgsql + + + * Now we need to make a database. Open the pgAdmin application that was installed with PostgreSQL. In the Browser tab, open Servers, then open PostgreSQL 9.6 (or whatever version was installed). This will prompt a login using the password you provided during PostgreSQL installation. After login, right-click Databases and create a database. I suggest giving it the name `icdataserver` and leaving everything else as default. Close pgAdmin. + * You need to configure a data source name (DSN) so that ICDataServer can connect to the database. Open ODBC Data Source Administrator (64-bit) which seems to be installed by default in Windows 10. Under the User DSN tab, add a new DSN with the PostgreSQL ODBC driver (Unicode if available). You'll want to enter the following details: + + * Data Source: `icdataserver` (this is the DSN name) + * Database: `icdataserver` (name of the database you created in pgAdmin) + * Server: `localhost` + * Port: `5432` (or whatever you chose during PostgreSQL installation) + * Username and password as installed. Ideally a different user would be created. + + + * Install ICDataServer: `Pkg.clone("https://github.com/painterqubits/ICDataServer.jl.git")` @@ -22,10 +41,28 @@ Automatic logging of measurement data and metadata. ## Quick start +You need to provide some configuration files that contain the default entries for the data server. This includes things like the users, servers and their locations, and instrument details. This is detailed in the Configuration docs. + + +On the first run of ICDataServer, you'll need to setup tables in the database to keep track of all the information you'll be submitting to it. Press enter/return when prompted for a path to the setup files, or you can provide another path. + + ``` julia> using ICDataServer -julia> ICDataServer.setuptables() +julia> IC = ICDataServer; dsn = IC.dsn; + +julia> IC.setuptables(dsn) +Path to directory with setup files? [defaults to ICDataServer/deps]: + +C:\Users\Discord\.julia\v0.5\ICDataServer\deps +``` + +Now the data server can be started. It should be running before you start using `InstrumentControl`. + + +``` +julia> IC.serve() # or @async IC.serve() to have it run in the background. ``` diff --git a/docs/build/instruments.md b/docs/build/instruments.md index 280f11b..3864e4e 100644 --- a/docs/build/instruments.md +++ b/docs/build/instruments.md @@ -16,7 +16,7 @@ newinstrument(dsn, alias; make="", model="", serialnum="", kind="", Create a new instrument named `alias` in the `instruments` table. -source
+source
# **`ICDataServer.updateinstrument`** — *Function*. @@ -30,7 +30,7 @@ updateinstrument(dsn, alias; kwargs...) Update any of the fields for an instrument named `alias`. Possible keyword arguments listed in documentation for [`ICDataServer.newinstrument`](instruments.md#ICDataServer.newinstrument). -source
+source
# **`ICDataServer.deleteinstrument`** — *Function*. @@ -44,7 +44,7 @@ deleteinstrument(dsn, alias) Delete instrument named `alias` from the `instruments` table. -source
+source
# **`ICDataServer.listinstruments`** — *Function*. @@ -58,7 +58,7 @@ listinstruments(dsn) List instruments in the `instruments` table. -source
+source
diff --git a/docs/build/jobs.md b/docs/build/jobs.md index 3ee9a41..baff1e2 100644 --- a/docs/build/jobs.md +++ b/docs/build/jobs.md @@ -1,22 +1,17 @@ - - - -## Jobs - # **`ICDataServer.newjob`** — *Function*. ``` -newjob(dsn, dataserver; cryostat="", uname="", device="", - nmeas=1, jobstart="", jobstop="", jobstatus=0) +newjob(dsn; cryostat="", username="", device="", + nmeas=1, jobstart="", jobstop="", jobstatus=0, dataserver="") ``` Create a new job in the `jobs` table. This function will return a `DataFrame` containing the columns `job_id` and `jobsubmit` with the inserted job id and job submission time. -source
+source
# **`ICDataServer.updatejob`** — *Function*. @@ -30,5 +25,5 @@ updatejob(dsn, job_id; kwargs...) Update an existing job in the `jobs` table based on its `job_id`. Specify the fields to update with keyword arguments specified in [`ICDataServer.newjob`](jobs.md#ICDataServer.newjob). -source
+source
diff --git a/docs/build/servers.md b/docs/build/servers.md index 1b381dd..11c9858 100644 --- a/docs/build/servers.md +++ b/docs/build/servers.md @@ -1,8 +1,3 @@ - - - -## Servers - # **`ICDataServer.newserver`** — *Function*. @@ -15,7 +10,7 @@ newserver(dsn, name, addr, port::Integer=-1) Create a new server in the `servers` table. -source
+source
# **`ICDataServer.updateserver`** — *Function*. @@ -26,10 +21,10 @@ Create a new server in the `servers` table. updateserver(dsn, name; kwargs...) ``` -Update a server in the `servers` table using keyword arguments (`name`, `addr`, `port`). +Update a server in the `servers` table using keyword arguments (`name`, `address`, `port`). -source
+source
# **`ICDataServer.deleteserver`** — *Function*. @@ -43,7 +38,7 @@ deleteserver(dsn, name) Remove a server from the `servers` table by passing its `name`. -source
+source
# **`ICDataServer.listservers`** — *Function*. @@ -57,5 +52,5 @@ listservers(dsn, name) List all servers in the `servers` table. -source
+source
diff --git a/docs/build/users.md b/docs/build/users.md index b98662e..b8232df 100644 --- a/docs/build/users.md +++ b/docs/build/users.md @@ -1,13 +1,56 @@ +# +**`ICDataServer.newuser`** — *Function*. - -## Users + +``` +newuser(dsn, username, name; email="", phone="", office="") +``` + +This function creates a new user in the `users` table of the database. E-mail, phone, office are useful for contacting users about their measurements. + + +source
+ +# +**`ICDataServer.updateuser`** — *Function*. + ``` -newuser -updateuser -deleteuser -listusers +updateuser(dsn, username; kwargs...) ``` +Update an existing user in the `users` table, identified by `username`. Specify the fields to update with keyword arguments specified in [`ICDataServer.newuser`](users.md#ICDataServer.newuser). + + +source
+ +# +**`ICDataServer.deleteuser`** — *Function*. + + + +``` +deleteuser(dsn, username) +``` + +Delete a user from the `users` table by providing the username. + + +source
+ +# +**`ICDataServer.listusers`** — *Function*. + + + +``` +listusers(dsn) +``` + +List all users in the `users` table. + + +source
+ diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 5f7dc7f..a0e57b0 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -27,7 +27,9 @@ docs_dir: 'build' pages: - Home: index.md - Design: design.md +- Configuration: config.md - API: + - 'General notes': api.md - 'Jobs': jobs.md - 'Users': users.md - 'Servers': servers.md diff --git a/docs/src/api.md b/docs/src/api.md new file mode 100644 index 0000000..6403192 --- /dev/null +++ b/docs/src/api.md @@ -0,0 +1,6 @@ +Many if not all of the functions detailed in this API are really just +convenience functions wrapped around methods in the +[`ODBC.jl`](https://github.com/JuliaDB/ODBC.jl) package, with some syntax +specific to how we have set up our databases. + +Often a `dsn` argument is requested. You can provide `ICDataServer.dsn`. diff --git a/docs/src/config.md b/docs/src/config.md new file mode 100644 index 0000000..0b4555c --- /dev/null +++ b/docs/src/config.md @@ -0,0 +1,92 @@ +Several configuration files should be generated and put into the `deps` directory +of the ICDataServer package. These are not tracked by git as they may contain +sensitive information and in any case are specific to the particular installation. +These are used by [`ICDataServer.setuptables`](@ref) to add default information. + +```@docs +ICDataServer.setuptables +``` + +### deps/config.json + +This JSON file consists of up to four keys and largely exists to provide +connection information between the database and InstrumentControl clients. +Typical example: + +``` +{ + "jobsock":"tcp://*:50001", + "dsn":"icdataserver", +} +``` + +Additional fields not shown in the example may include: + +- "username": A username to connect to the database, if not provided when the + DSN was configured. +- "password": A password to connect to the database, if not provided when the + DSN was configured. + +### deps/instrumentkinds.json + +This JSON file declares the various kinds of instruments. Its purpose is to +standardize such declarations in the database. Typical example: + +``` +{ + "kind": [ + "awg", + "dcsource", + "rfsource", + "vna" + ] +} +``` + +### deps/servers.json + +Declaration of servers. Typical example: + +``` +{ + "servers": [ + { + "alias":"local_data", + "address":"127.0.0.1", + "port":"50002" + }, + { + "alias":"local_hw", + "address":"127.0.0.1", + "port":"50003" + } + ] +} +``` + +### deps/users.json + +Declaration of users, including contact information which may be used by +the software to send alerts and notifications. Typical example: + +``` +{ + "users": [ + { + "username":"default", + "first":"No", + "last":"Name" + }, { + "username":"anotheruser", + "first":"Really", + "middle":"No", + "last":"Name", + "email":"someaddress@caltech.edu", + "phone":"15555551212", + "office":"Watson 5th floor" + } + ] +} +``` + +The only required fields for each dictionary are `username`, `first`, `last`. diff --git a/docs/src/design.md b/docs/src/design.md index 9881f9a..0aca36c 100644 --- a/docs/src/design.md +++ b/docs/src/design.md @@ -1,3 +1 @@ -## Design - ICDataServer.jl diff --git a/docs/src/index.md b/docs/src/index.md index b919348..e8409fc 100644 --- a/docs/src/index.md +++ b/docs/src/index.md @@ -4,19 +4,67 @@ Automatic logging of measurement data and metadata. ## Installation -If you plan to run a database: -+ Install the most recent version of [PostgreSQL](https://www.postgresql.org/download/) -+ Install the 64-bit ODBC driver for PostgreSQL - (on Windows, it is an optional installation prompted automatically). +Estimated burden: twenty minutes. -Whether you plan to run a database or not: -+ `Pkg.clone("https://github.com/painterqubits/ICDataServer.jl.git")` ++ Install the most recent version of [PostgreSQL](https://www.postgresql.org/download/). + When setting up, a password for the default user (`postgres`) will be requested. + Choose something secure. + ++ Following the installation, Stack Builder will open. This lets you add + components to the PostgreSQL installation. Install from Database Drivers + (64-bit when there is an option): + + + ODBC driver (only one of the three that is currently utilized) + + JDBC driver + + Npgsql + + ++ Now we need to make a database. Open the pgAdmin application that was + installed with PostgreSQL. In the Browser tab, open Servers, then open + PostgreSQL 9.6 (or whatever version was installed). This will prompt a login + using the password you provided during PostgreSQL installation. After login, + right-click Databases and create a database. I suggest giving it the name + `icdataserver` and leaving everything else as default. Close pgAdmin. + ++ You need to configure a data source name (DSN) so that ICDataServer can + connect to the database. Open ODBC Data Source Administrator (64-bit) which + seems to be installed by default in Windows 10. Under the User DSN tab, add + a new DSN with the PostgreSQL ODBC driver (Unicode if available). You'll want + to enter the following details: + + + Data Source: `icdataserver` (this is the DSN name) + + Database: `icdataserver` (name of the database you created in pgAdmin) + + Server: `localhost` + + Port: `5432` (or whatever you chose during PostgreSQL installation) + + Username and password as installed. Ideally a different user would be created. + + ++ Install ICDataServer: `Pkg.clone("https://github.com/painterqubits/ICDataServer.jl.git")` ## Quick start +You need to provide some configuration files that contain the default entries +for the data server. This includes things like the users, servers and their +locations, and instrument details. This is detailed in the Configuration docs. + +On the first run of ICDataServer, you'll need to setup tables in the database +to keep track of all the information you'll be submitting to it. Press enter/return +when prompted for a path to the setup files, or you can provide another path. + ``` julia> using ICDataServer -julia> ICDataServer.setuptables() +julia> IC = ICDataServer; dsn = IC.dsn; +julia> IC.setuptables(dsn) +Path to directory with setup files? [defaults to ICDataServer/deps]: + +C:\Users\Discord\.julia\v0.5\ICDataServer\deps +``` + +Now the data server can be started. It should be running before you start +using `InstrumentControl`. + +``` +julia> IC.serve() # or @async IC.serve() to have it run in the background. ``` diff --git a/docs/src/instruments.md b/docs/src/instruments.md index bfd3633..00f6b3d 100644 --- a/docs/src/instruments.md +++ b/docs/src/instruments.md @@ -1,10 +1,10 @@ ## Instruments ```@docs -newinstrument -updateinstrument -deleteinstrument -listinstruments +ICDataServer.newinstrument +ICDataServer.updateinstrument +ICDataServer.deleteinstrument +ICDataServer.listinstruments ``` ## Constraints on the `instruments` table diff --git a/docs/src/jobs.md b/docs/src/jobs.md index 0af5ab2..bdac071 100644 --- a/docs/src/jobs.md +++ b/docs/src/jobs.md @@ -1,6 +1,4 @@ -## Jobs - ```@docs -newjob -updatejob +ICDataServer.newjob +ICDataServer.updatejob ``` diff --git a/docs/src/servers.md b/docs/src/servers.md index 9aa4e80..a869527 100644 --- a/docs/src/servers.md +++ b/docs/src/servers.md @@ -1,8 +1,6 @@ -## Servers - ```@docs -newserver -updateserver -deleteserver -listservers +ICDataServer.newserver +ICDataServer.updateserver +ICDataServer.deleteserver +ICDataServer.listservers ``` diff --git a/docs/src/users.md b/docs/src/users.md index 2c9976e..c220691 100644 --- a/docs/src/users.md +++ b/docs/src/users.md @@ -1,8 +1,6 @@ -## Users - ```@docs -newuser -updateuser -deleteuser -listusers +ICDataServer.newuser +ICDataServer.updateuser +ICDataServer.deleteuser +ICDataServer.listusers ``` diff --git a/src/setup.jl b/src/setup.jl index 70924fa..ad2004e 100644 --- a/src/setup.jl +++ b/src/setup.jl @@ -14,9 +14,9 @@ Initialize a database with the tables needed to run ICDataServer: The creation of the tables can proceed without additional input, but in some cases it may make sense for the tables to be automatically filled with data. -If new instrument kinds are added to `deps/inskinds.json`, they will be added -to `instrumentkinds` table. If kinds are encountered in the database that are -not in the file, they remain in the database and are not deleted. +If new instrument kinds are added to `deps/instrumentkinds.json`, they will be +added to `instrumentkinds` table. If kinds are encountered in the database that +are not in the file, they remain in the database and are not deleted. If new users are added to `deps/users.json`, they will be "upserted" into the `users` table (users are created if necessary; if an existing user name is