Skip to content

Defining, building, deploying a microservice

Jump to bottom
Emad Aghayi edited this page Apr 19, 2019 · 7 revisions

Crowd Microservices is the first microtask crowdsourcing system for implementing microservices, a novel behavior-driven development workflow for crowdsourcing programming which offers workers immediate feedback on their contributions, and a cloud-based IDE for building microservices. Crowd Microservices follows client-server architecture with three layers: 1) a web client, implemented in AngularJS, which runs on a worker's browser, 2) a back-end, implemented in Node.js, and 3) a persistence store, implemented using Firebase Real-time Database. In the rest of this document, we provide a brief description of Crowd Microservices.

Client-Request

Crowd Microservices applies the behavior-driven development workflow to implementing microservices. The microservice is first described by a client through Client-Request page. Clients define a set of endpoints describing HTTP requests which will be handled by the microservice. Each endpoint is defined as a function, specifying an identifier, parameters, and a description of its behavior. As endpoints may accept complex JSON data structures as input and generate complex JSON data structures as output, clients may also describe a set of abstract data types (ADTs). Each ADT describes a set of fields for a JSON object, assigning each field a type which may be either a primitive or another ADT. In defining endpoints, clients may specify the expected data by giving each parameter and return value a type.

In the bellow we provided some points that might be useful for working with Client-Request page:

  • For interacting with Client-request page click on the link: http://crowdcode5.herokuapp.com/clientRequest . Since it needs authentication, in the first landing it redirects you to authentication page, so after you logged in successfully you might click again on the above link again for visiting the Client-Request (In next version we try to integrate authentication page and client request in one page to avoid twice clicking to reach the Client-request page ).

  • All buttons and fields have description to give the information to the admin user for working with Client-request, in addition we attached a filled Clint-Request PDF sample attached to this document. This sample might help you to figuring out how to define the endpoints, ADTs.

  • Please pay attention the ADTs are JSON format. Also endpoints are only a name they do not follow the endpoint format like GET /user/info, in the client request you only define. For example if define the endpoint name as “fetchTodoBasedOnStatus” and parameters like “userId”and “status”, in the final deployed endpoints the microservice will be “/fetchTodoBasedOnStatus” and the “req.query.userId” and “req.query.status” will be the parameter of this endpoint. You can see the defining endpoint and the final output of endpoint which will be deployed on a GitHub repo in the below codes.

        `router.get('/fetchTodosBasedOnStatus', async (req, res, next) => {`
         `const todoList = await service.fetchTodosBasedOnStatus(req.query.userId, req.query.status);`
         `res.json(todoList);`
       `});`

  • Third party APIs checkboxes are for 5 predefined functions. Through this API, workers can store, update, and delete JSON objects in a persistence store. Workers may use any of these API functions when working with functions and unit tests in the Implement Function Behavior microtask. Any schema-less persistence store may be used as an implementation for this API. In our IDE, a development version used by workers simulates the behavior of a persistence store within the browser and clears the persistence store after every test execution. In fact, we implemented a sandbox environment for testing in our IDE. In the production version used after the microservice is deployed, the persistence API is implemented through a Firebase store. Therefore, for using third-party APIs for persistence store, you must create functions and parameters like the name and parameters are provided in the attached sample PDF, then click on the checkboxes. I also provide their names and parameters here. These 5 functions are provided for working with todo JSON objects if you want to pass other objects rather than todo objects let us know for changing their implementation in the Crowd Microservices. SaveObject It inserts an object in the database. It returns 'Duplicate' if it is already persisted in the database. Parameters todo Todo Return: Todo`

     `FetchObject`
 `Fetch the object from the database. It returns null if the desired object does not exist.`
 `Parameters`
 `objectId Number`
 `Return: Todo`

 `DeleteObjectImplementation`
 `Delete an object from the database. It returns null if the desired object does not exist.`
 `Parameters`
 `todo Todo`
 `Return: Todo`

 `UpdateObject`
 `Update an object in the database. It returns null if the desired object does not exist.`
 `Parameters`
 `todo Todo`
 `Return: Todo`

 `FetchAllObjects`
 `Get the list of objects in the database.`
 `Parameters`
 `userId String`
 `Return: Todo[]`

*After you have finished defining the microservice in the Client-page, the project is created. After you landed in the http://crowdcode5.herokuapp.com/ , In the welcome page at the end Crowd Microservices has a "Get Started" button which route you to the defined project which you defined in the client request (the name you choose in the first input text in the Client-request as your project name) . For instance, you created a project with name “todo” in your client request, so when you click on the "Get Started" button, you must route to URL like http://crowdcode5.herokuapp.com/todo . The pattern would be http://crowdcode5.herokuapp.com/_PROJECT_NAME . Every time you create a new project in the Client-request you must correct the URL that “Get Started” button routes you. *For working with the system, you need at least two workers. It makes sense for improving the quality of code and avoiding malicious activity the contribution reviewed by another worker. When testing the environment, we usually simulate two workers by using two browsers with two separate accounts.

Microtasks

After a client has completed a client request, they may then submit this client request to generate a new Crowd Microservices project. Submitting a client request generates an initial set of microtasks, generating an Implement Function Behavior microtask for each endpoint function. The crowd may then log into the project to begin completing microtasks. As workers complete microtasks, additional microtasks are automatically generated by the system to review contributions, continue work on each function, and implement any new functions requested by crowd workers. Crowd Microservices automatically generates microtasks based on the current state of submitted work. After a client request defines endpoints, the system automatically generates a function and microtask to begin work on each. After an Implement Function Behavior microtask is submitted, the system automatically creates a Review microtask. After a Review microtask is submitted, an Implement Function Behavior is generated to continue work, if the contributor has not indicated that work is complete. If a review of an Implement Function Behavior contribution indicates issues that need to be fixed, a new Implement Function Behavior microtask is generated, which includes the issue and instruction to fix it. After a microtask is generated, it is added to a queue. When a worker fetches a microtask, the system automatically assigns the worker the next microtask and removes it from the queue.

Microservices Deployment

After the crowd finishes the implementation of a microservice, the client may choose to create and deploy the microservice to a hosting. Invoking the Publish command first creates a new node.js GitHub project which includes each function implemented by the crowd. For endpoint functions, the environment automatically generates an HTTP request handler function for the endpoint. Each endpoint then contains the implementation of the function defined by the crowd. Next, this GitHub project is deployed to a hosting site. In Crowd Microservices, projects are deployed to the Heroku hosting site. A new project instance is created, and the project deployed. After this process has been completed, the client may then begin using the completed microservice by making HTTP requests against the deployed, publicly available microservice. In the bellow we provided some points that might be useful for working with Deployment:

  • After you finished defining the endpoints in the client request, Crowd Microservices needs to be restarted. I can restart the server, which is hosted on Heroku, or I can give you permission for doing that yourself by GUI dashboard in Heroku. If you want to do it yourself, you have to give me the user name that you signed up in the Heroku. https://devcenter.heroku.com/
  • After the crowd finishes the implementation of a microservice, you can deploy their contribution as a microservice on a GitHub repo. For doing this step, we give you a template Node.js project (similar to https://github.com/eaghayi/crowd-todo-microservices) to create as an empty project on your GitHub account. For pushing the code by the Crowd Microservices, it needs your GitHub user name, repository name, and a token. You might send them to us, and we will add your information as a valid user in Crowd Microservices.
  • Next, for deploying the implemented function by the crowd on the GitHub, you need to run a command by clicking on the link: http://crowdcode5.herokuapp.com/{project_name_you_selected_in_client_Request}/deploy .It will deploy the implemented code by the crowd on a GitHub repo.

Resource

You can find useful information in the below resources.

  1. Paper: http://mason.gmu.edu/~eaghayi/papers/Eaghayi2019TSE.pdf
  2. Demo video on Crowd Microservices: https://youtu.be/qQeYOsRaxHc
  3. Crowd Microservices URL: http://crowdcode5.herokuapp.com/
  4. Client-Request URL: http://crowdcode5.herokuapp.com/clientRequest

devuxd laboratory

Clone this wiki locally