Add curl option

RELEASE.md

@@ -23,6 +23,13 @@ files respectively, in the configure/ directory of the appropriate release of th
 Release Notes
 =============
 
+R2-4 (22-Aug-2024)
+----
+* Added header file and transferred class declaration to there.
+* Added curl usage: driver can use either curl or GraphicksMagick to get image from URL.
+* Added WITH_CURL compilation option to compile with or without libcurl.
+* Add [documentation](docs/ADURL/ADURL.rst) about curl configuration records.
+
 R2-3 (26-May-2021)
 ----
 * Converted documentation to ReST, include in documentation on github.io.

configure/CONFIG_SITE

@@ -15,6 +15,7 @@
 # Set CHECK_RELEASE to WARN to perform consistency checking but
 #   continue building anyway if conflicts are found.
 CHECK_RELEASE = YES
+# WITH_CURL = YES
 
 # Set this when you only want to compile this application
 #   for a subset of the cross-compiled target architectures

docs/ADURL/ADURL.rst

@@ -12,15 +12,30 @@ Introduction
 
 This is an :doc:`../index` driver for reading images from a
 URL. It can be used to read images from Web cameras, `Axis video
-servers`_, or simply from a disk file. It reads the images using 
-`GraphicsMagick`_ and can thus read images encoded in any of the
-formats supported by GraphicsMagick, such as JPEG, TIFF, PNG, etc.
+servers`_, or simply from a disk file. It can read the images
+using either `GraphicsMagick`_ or `Curl`_ (if compiled with the
+WITH_CURL=YES option on CONFIG_SITE and toggling the UseCurl record)
+and can thus read images encoded in any of the formats supported
+by GraphicsMagick, such as JPEG, TIFF, PNG, etc.
 
 The driver simply reads images from the specified URL at the rate
 determined by the AcquirePeriod. Web cameras and Axis video servers
 have a URL address from which the current image can be read. There are
 often several addresses for different image sizes.
 
+If configured correctly, curl can support both url redirection and authentication.
+To avoid tedious, repetitive and error prone configuration processes, curl can
+have a configuration file with all `curl options`_ in the format key = value. Example:
+
+| CURLOPT_HTTPAUTH = 2
+| CURLOPT_USERNAME = User
+| CURLOPT_PASSWORD = PassWord
+
+Each curl option must have a value adequate to the asynParamType that the option
+is implemented. For example, CURLOPT_HTTPAUTH = CURLAUTH_BASIC will not work. Also,
+the file only configure asyn-implemented options. Unfortunately, it does not configure
+curl options that are not yet implemented in the driver.
+
 This driver inherits from :doc:`../ADCore/ADDriver`. It implements many 
 of the parameters in `asynNDArrayDriver.h`_ and in `ADArrayDriver.h`_. It also
 implements a number of parameters that are specific to the URL
@@ -54,6 +69,24 @@ the standard driver parameters.
       **NOTE: If this value is set to 0 or too small a value can
       result in the driver using 100% of the CPU and becoming unresponsive
       to EPICS.**
+  * - NDFilePath
+    - $(P)$(R)CfgFilePath
+    - Path of directory where to find curl configuration file.
+  * - NDFilePathExists
+    - $(P)$(R)CfgFilePathExists_RBV
+    - Is set to one if driver thinks the file exists. Is set to 0 otherwise.
+  * - NDFileName
+    - $(P)$(R)CfgFileName
+    - Curl configuration file name. The file should contain values to be put
+      into each record in the format:
+      key = value
+      Example:
+      CURLOPT_HTTPAUTH = 2
+      CURLOPT_USERNAME = User
+      CURLOPT_PASSWORD = PassWord
+  * - NDFullFileName
+    - $(P)$(R)CfgFullFileName_RBV
+    - Full filename appended with filepath.
 
 
 URL driver specific parameters
@@ -79,6 +112,35 @@ those in asynNDArrayDriver.h and ADDriver.h.
   * - Selects which os the 10 URLs to read from. 
     - $(P)$(R)URLSelect
     - mbbo
+  * - Determines if image is acquired from URL with GraphicsMagick or Curl.
+    - $(P)$(R)UseCurl
+    - bo
+  * - Sets curl's HTTP authentication method.
+    - $(P)$(R)CurlOptHTTPAuth, $(P)$(R)CurlOptHTTP_RBV
+    - mbbo, mbbi
+  * - Toggle curl's hostname verification in SSL certificate.
+    - $(P)$(R)CurlOptSSLVerifyHost, $(P)$(R)CurlOptSSLVerifyHost_RBV
+    - mbbo, mbbi
+  * - Toggle curl's SSL certificate verification.
+    - $(P)$(R)CurlOptSSLVerifyPerr, $(P)$(R)CurlOptSSLVerifyPeer_RBV
+    - bo, bi
+  * - Set curl's authentication username.
+    - $(P)$(R)CurlOptUserName
+    - waveform
+  * - Set curl's authentication password.
+    - $(P)$(R)CurlOptPassword
+    - waveform
+  * - Shows if driver has read permission to file.
+    - $(P)$(R)CfgFileValid_RBV
+    - bi
+  * - $(P)$(R)CurlLoadConfig
+    - Loads configuration from file in $(P)$(R)CfgFullFileName_RBV. BEWARE:
+      this will probably make the setpoint and readback curl option records have
+      different values. For example, if you set $(P)$(R)CurlOptHTTPAuth and then
+      load a CurlConfigFile that changes it, $(P)$(R)CurlOptHTTPAuth_RBV is going
+      to have a different file. Also, $(P)$(R)CurlOptUserName and
+      $(P)$(R)CurlOptUserPassword are going to be set in the curl option, but there
+      is no readback for them.
 
 The URLs for Web cameras and video servers are typically long strings,
 which are difficult to remember and to type. Thus, for convenience
@@ -174,5 +236,6 @@ connected to analog camera through an Axis video server.
 .. _GraphicsMagick: http://www.graphicsmagick.org/
 .. _ADDriver: ../areaDetectorDoc.html#ADDriver
 .. _Axis video servers: http://www.axis.com/
-
+.. _Curl: https://curl.se/
+.. _curl options: https://curl.se/libcurl/c/curl_easy_setopt.html
 

iocs/urlIOC/iocBoot/iocURLDriver/st_base.cmd

@@ -28,6 +28,11 @@ epicsEnvSet("EPICS_DB_INCLUDE_PATH", "$(ADCORE)/db")
 URLDriverConfig("$(PORT)", 0, 0)
 dbLoadRecords("$(ADURL)/db/URLDriver.template","P=$(PREFIX),R=cam1:,PORT=$(PORT),ADDR=0,TIMEOUT=1")
 
+#### If using curl uncomment this
+## CurlUsage.template includes another template file under $(ADURL)/db/
+# epicsEnvSet("EPICS_DB_INCLUDE_PATH", "$(EPICS_DB_INCLUDE_PATH):$(ADURL)/db/")
+# Load curl configuration records
+# dbLoadRecords("$(ADURL)/db/CurlUsage.template","P=$(PREFIX),R=cam1:,PORT=$(PORT),ADDR=0,TIMEOUT=1")
 # Create a standard arrays plugin.
 NDStdArraysConfigure("Image1", 3, 0, "$(PORT)", 0)
 

urlApp/Db/CurlConfigFile.template

@@ -0,0 +1,91 @@
+#=================================================================#
+# Template file: CurlConfigFile.template
+# Made to load files. Inspired but not completely copied from NDFile.template
+# Marco Montevechi
+# aug 11, 2024
+
+###################################################################
+#  These records control Config file loading                      #
+###################################################################
+
+# File path.
+record(waveform, "$(P)$(R)CfgFilePath")
+{
+    field(PINI, "YES")
+    field(DTYP, "asynOctetWrite")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_PATH")
+    field(FTVL, "CHAR")
+    field(NELM, "256")
+    info(autosaveFields, "VAL")
+}
+
+record(waveform, "$(P)$(R)CfgFilePath_RBV")
+{
+    field(DTYP, "asynOctetRead")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_PATH")
+    field(FTVL, "CHAR")
+    field(NELM, "256")
+    field(SCAN, "I/O Intr")
+}
+
+record(bi, "$(P)$(R)CfgFilePathExists_RBV")
+{
+    field(DTYP, "asynInt32")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_PATH_EXISTS")
+    field(ZNAM, "No")
+    field(ZSV,  "MAJOR")
+    field(ONAM, "Yes")
+    field(OSV,  "NO_ALARM")
+    field(SCAN, "I/O Intr")
+}
+
+# Filename
+record(waveform, "$(P)$(R)CfgFileName")
+{
+    field(PINI, "YES")
+    field(DTYP, "asynOctetWrite")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_NAME")
+    field(FTVL, "CHAR")
+    field(NELM, "256")
+    info(autosaveFields, "VAL")
+}
+
+record(waveform, "$(P)$(R)CfgFileName_RBV")
+{
+    field(DTYP, "asynOctetRead")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_NAME")
+    field(FTVL, "CHAR")
+    field(NELM, "256")
+    field(SCAN, "I/O Intr")
+}
+
+# Full filename, including path
+record(waveform, "$(P)$(R)CfgFullFileName_RBV")
+{
+    field(DTYP, "asynOctetRead")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FULL_FILE_NAME")
+    field(FTVL, "CHAR")
+    field(NELM, "512")
+    field(SCAN, "I/O Intr")
+}
+
+# Full filename, including path
+record(bi, "$(P)$(R)CfgFileValid_RBV")
+{
+    field(DTYP, "asynInt32")
+    field(INP,  "@asyn($(PORT),$(ADDR=0),$(TIMEOUT=1))FILE_IS_VALID")
+    field(ZNAM, "No")
+    field(ZSV,  "MAJOR")
+    field(ONAM, "Yes")
+    field(OSV,  "NO_ALARM")
+    field(SCAN, "I/O Intr")
+}
+
+record (bo, "$(P)$(R)CurlLoadConfig")
+{
+    field(DESC, "Load curl configuration from file")
+    field(DTYP, "asynInt32")
+    field(ZNAM, "0")
+    field(ONAM, "1")
+    field(OUT,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))CURL_LOAD_CONFIG")
+}

urlApp/Db/CurlUsage.template

@@ -0,0 +1,139 @@
+# Records to toggle curl functionalities
+
+include "CurlConfigFile.template"
+
+record (bo, "$(P)$(R)UseCurl")
+{
+    field(DESC, "Toggle curl usage")
+    field(DTYP, "asynInt32")
+    field(PINI, "YES")
+    field(ZNAM, "NO")
+    field(ONAM, "YES")
+    field(OUT,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))USE_CURL")
+    info(autosaveFields, "VAL")
+}
+
+record (mbbo, "$(P)$(R)CurlOptHTTPAuth")
+{
+    field(DESC, "Set CURLOPT_HTTPAUTH")
+    field(DTYP, "asynInt32")
+    field(ZRST, "CURLAUTH_BASIC")
+    field(ZRVL, "0")
+    field(ONST, "CURLAUTH_DIGEST")
+    field(ONVL, "1")
+    field(TWST, "CURLAUTH_DIGEST_IE")
+    field(TWVL, "2")
+    field(THST, "CURLAUTH_BEARER")
+    field(THVL, "3")
+    field(FRST, "CURLAUTH_NEGOTIATE")
+    field(FRVL, "4")
+    field(FVST, "CURLAUTH_NTLM")
+    field(FVVL, "5")
+    field(SXST, "CURLAUTH_NTLM_WB")
+    field(SXVL, "6")
+    field(SVST, "CURLAUTH_ANY")
+    field(SVVL, "7")
+    field(EIST, "CURLAUTH_ANYSAFE")
+    field(EIVL, "8")
+    field(NIST, "CURLAUTH_ONLY")
+    field(NIVL, "9")
+    field(TEST, "CURLAUTH_AWS_SIGV4")
+    field(TEVL, "10")
+    field(OUT,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_HTTPAUTH")
+}
+
+record (mbbi, "$(P)$(R)CurlOptHTTPAuth_RBV")
+{
+    field(DESC, "Set CURLOPT_HTTPAUTH")
+    field(DTYP, "asynInt32")
+    field(ZRST, "CURLAUTH_BASIC")
+    field(ZRVL, "0")
+    field(ONST, "CURLAUTH_DIGEST")
+    field(ONVL, "1")
+    field(TWST, "CURLAUTH_DIGEST_IE")
+    field(TWVL, "2")
+    field(THST, "CURLAUTH_BEARER")
+    field(THVL, "3")
+    field(FRST, "CURLAUTH_NEGOTIATE")
+    field(FRVL, "4")
+    field(FVST, "CURLAUTH_NTLM")
+    field(FVVL, "5")
+    field(SXST, "CURLAUTH_NTLM_WB")
+    field(SXVL, "6")
+    field(SVST, "CURLAUTH_ANY")
+    field(SVVL, "7")
+    field(EIST, "CURLAUTH_ANYSAFE")
+    field(EIVL, "8")
+    field(NIST, "CURLAUTH_ONLY")
+    field(NIVL, "9")
+    field(TEST, "CURLAUTH_AWS_SIGV4")
+    field(TEVL, "10")
+    field(SCAN, "I/O Intr")
+    field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_HTTPAUTH")
+}
+
+record (mbbo, "$(P)$(R)CurlOptSSLVerifyHost")
+{
+    field(DESC, "Verify Host SSL certificate")
+    field(DTYP, "asynInt32")
+    field(ZRST, "NO")
+    field(ZRVL, "0")
+    field(ONST, "YES")
+    field(ONVL, "1")
+    field(TWST, "YES")
+    field(TWVL, "2")
+    field(OUT,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_SSL_VERIFYHOST")
+}
+
+record (mbbi, "$(P)$(R)CurlOptSSLVerifyHost_RBV")
+{
+    field(DESC, "Verify Host SSL certificate")
+    field(DTYP, "asynInt32")
+    field(ZRST, "NO")
+    field(ZRVL, "0")
+    field(ONST, "YES")
+    field(ONVL, "1")
+    field(TWST, "YES")
+    field(TWVL, "2")
+    field(SCAN, "I/O Intr")
+    field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_SSL_VERIFYHOST")
+}
+
+record (bo, "$(P)$(R)CurlOptSSLVerifyPeer")
+{
+    field(DESC, "Verify SSL peer")
+    field(DTYP, "asynInt32")
+    field(PINI, "YES")
+    field(ZNAM, "NO")
+    field(ONAM, "YES")
+    field(OUT,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_SSL_VERIFYPEER")
+}
+
+record (bi, "$(P)$(R)CurlOptSSLVerifyPeer_RBV")
+{
+    field(DESC, "Verify SSL peer")
+    field(DTYP, "asynInt32")
+    field(PINI, "YES")
+    field(ZNAM, "NO")
+    field(ONAM, "YES")
+    field(SCAN, "I/O Intr")
+    field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_SSL_VERIFYPEER")
+}
+
+record(waveform, "$(P)$(R)CurlOptUserName")
+{
+    field(DESC, "Username for auth. Try not to use this.")
+    field(DTYP, "asynOctetWrite")
+    field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_USERNAME")
+    field(FTVL, "CHAR")
+    field(NELM, "128")
+}
+
+record(waveform, "$(P)$(R)CurlOptPassword")
+{
+    field(DESC, "Password for auth. Try not to use this.")
+    field(DTYP, "asynOctetWrite")
+    field(INP,  "@asyn($(PORT),$(ADDR),$(TIMEOUT))ASYN_CURLOPT_PASSWORD")
+    field(FTVL, "CHAR")
+    field(NELM, "128")
+}

urlApp/Db/Makefile

@@ -12,6 +12,10 @@ include $(TOP)/configure/CONFIG
 # databases, templates, substitutions like this
 
 DB += URLDriver.template
+ifeq ($(WITH_CURL), YES)
+	DB += CurlUsage.template
+	DB += CurlConfigFile.template
+endif
 
 #----------------------------------------------------
 # If <anyname>.db template is not named <anyname>*.template add

urlApp/src/Makefile

@@ -43,6 +43,11 @@ ifeq ($(WITH_GRAPHICSMAGICK), YES)
   include $(ADCORE)/ADApp/commonLibraryMakefile
 endif
 
+ifeq ($(WITH_CURL),YES)
+  USR_CXXFLAGS += -DADURL_USE_CURL
+  USR_SYS_LIBS += curl
+endif
+
 #=============================
 
 include $(TOP)/configure/RULES