Legal Case Recommender - USA

How are the legal case data indexed and embeddings created?

Legal case data is downloaded from the United States Reports, the official reporter of the Supreme Court of the United States.
Data includes full texts of all cases from the inception of the Supreme Court to the present.
The data is preprocessed by eliminating special characters, stop words, and applying stemming to the words.
Data is then indexed using LangChain’s RecursiveCharacterTextSplitter with chunk_size as 1000, chunk_overlap as 100, and length_function as len.
Embeddings are generated using the OpenAI text-embedding-ada-002 model and have a dimension of 1536.
These embeddings are then safely stored in a database optimized for high-dimensional data, ensuring efficient storage and retrieval.

How do we keep the data updated?

Our system is linked directly with the United States Reports. We have an ongoing process that checks for newly published cases or updates to existing ones every week. New and updated data is preprocessed, indexed, and embeddings are generated before being stored in our vector database. This approach ensures our AI embeddings always have the latest legal case data.

Python Code

You can access the embedding using EmbedElite python package or curl request.

# This returns a list of embeddings
from embedelite import load_embedding

embeddings = load_embedding("legal-case-recommender-usa")
print(len(embeddings))
print(embeddings[1])

# This returns an object which can be directly inserted into Qdrant
result = load_embedding("legal-case-recommender-usa", embed_for="qdrant")
# result is {"embeddings": [], "documents": [] "ids": []}
print(result["embeddings"])
print(result["documents"])
print(result["ids"])

Curl

Request

curl -X POST -H "Content-Type: application/json" -d '{
"doc_id": "legal-case-recommender-usa"
}' https://api.embedelite.com/v1/embeddings/download/

Response

{
    "mappings": {
        "properties": {
        "doc_source": {"type": "keyword"},
        "sentence": { "type": "text" },
        "embeddings": { "type": "dense_vector", "dims": 1536, "index": False },
        "doc_source": {"type": "keyword"}
        }
    }

Data credit to the United States Reports, without which this embedding wouldn’t be possible.

Legal