feat: Part 4 (Final) - Introduce Main RAG Service API and its tests

[ishaansehgal99]

Reason for Change:
This series of PR will integrate llamaindex RAG service for Kaito.

This PR introduces the main API and its endpoints.

main.py introduces the main API
Three endpoints are introduced:

1. POST /index

Description:
Indexes a list of documents into the specified index.

Request Body (IndexRequest):

• index_name (str): The name of the index.
• documents (List[Document]):
  • text (str): The document's content.
  • metadata (Optional[dict]): Additional metadata (default: empty).

Response (List[DocumentResponse]):

• A list of indexed documents:
  • doc_id (str): The generated document ID.
  • text (str): The document content.
  • metadata (Optional[dict]): Document metadata.

2. POST /query

Description:
Queries the specified index and returns relevant results.

Request Body (QueryRequest):

• index_name (str): The index to query.
• query (str): The search query.
• top_k (int, default=10): The number of top results to return.
• llm_params (Optional[Dict]): Parameters for LLM processing.

Response (QueryResponse):

• response (str): The result or completion from the API.
• source_nodes (List[NodeWithScore]):
  • node_id (str): The node ID.
  • text (str): The node content.
  • score (float): The relevance score.
  • metadata (Optional[dict]): Node metadata.
• metadata (Optional[dict]): Query metadata.

3. GET /indexed-documents

Description:
Lists all documents currently indexed.

Response (ListDocumentsResponse):

• documents (Dict[str, Dict[str, Dict[str, str]]]):
  • Key: index_name
  • Value: A dictionary of documents in that index, where:
    • doc_id: The document ID.
    • text: The document content.

4. Additional Files

• config.py introduces the configurable params passed via environment variables from the ragengine controller

• models.py specifies the schema required for valid HTTP requests to the endpoints specified in main.py

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 52.85%. Comparing base (5c30038) to head (3d6a623).
Report is 42 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #603      +/-   ##
==========================================
- Coverage   58.18%   52.85%   -5.34%     
==========================================
  Files          30       34       +4     
  Lines        2987     4221    +1234     
==========================================
+ Hits         1738     2231     +493     
- Misses       1149     1870     +721     
- Partials      100      120      +20     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.