Financial Market Trend Analysis - USA

How is the financial data indexed and embeddings created?

• Financial data is sourced from major U.S. financial indices like the S&P 500, Dow Jones, NASDAQ, and more.
• The data is preprocessed by normalizing numerical values and identifying key trends and patterns.
• Data is then indexed using LangChain’s RecursiveCharacterTextSplitter with chunk_size as 1000, chunk_overlap as 100 and length_function as len.
• Embeddings are created using OpenAI text-embedding-ada-002 model and have a dimension of 1536.
• These embeddings are then safely stored in a database optimized for high-dimensional data, ensuring efficient storage and retrieval.

How do we keep the data updated?

Our system is integrated with major financial indices data streams. Every day, a process runs to check for new data or updates to existing data. New and updated data is preprocessed, indexed, and embeddings are generated before it’s stored in our vector database. This ensures our AI embeddings always have the most recent financial market data.

How to use

• Python
• Curl

Python Code

You can access the embedding using EmbedElite python package or curl request.

# this returns a list of embeddings
from embedelite import load_embedding

embeddings = load_embedding("financial-market-trends-usa")
print(len(embeddings))
print(embeddings[1])

# this returns an object which can be directly inserted into Qdrant
result = load_embedding("financial-market-trends-usa", embed_for="qdrant")
# result is {"embeddings": [], "documents": [] "ids": []}
print(result["embeddings"])
print(result["documents"])
print(result["ids"])

Curl

Request

curl -X POST -H "Content-Type: application/json" -d '{
"doc_id": "financial-market-trends-usa"
}' https://api.embedelite.com/v1/embeddings/download/

Response

{
    "mappings": {
        "properties": {
        "doc_source": {"type": "keyword"},
        "sentence": { "type": "text" },
        "embeddings": { "type": "dense_vector", "dims": 1536, "index": False },
        "doc_source": {"type": "keyword"}
        }
    }