Topic Modeling#

You’ll learn to use text embeddings to find text similarity and use that to create topics automatically from text, covering:

• Embeddings: How large language models convert text into numerical representations.
• Similarity Measurement: Understanding how similar embeddings indicate similar meanings.
• Embedding Visualization: Using tools like Tensorflow Projector to visualize embedding spaces.
• Embedding Applications: Using embeddings for tasks like classification and clustering.
• OpenAI Embeddings: Using OpenAI’s API to generate embeddings for text.
• Model Comparison: Exploring different embedding models and their strengths and weaknesses.
• Cosine Similarity: Calculating cosine similarity between embeddings for more reliable similarity measures.
• Embedding Cost: Understanding the cost of generating embeddings using OpenAI’s API.
• Embedding Range: Understanding the range of values in embeddings and their significance.

Here are the links used in the video:

• Jupyter Notebook
• Tensorflow projector
• Embeddings guide
• Embeddings reference
• Clustering on scikit-learn
• Massive text embedding leaderboard (MTEB)
• gte-large-en-v1.5 embedding model
• Embeddings similarity threshold