vipin wiki

Visualizing Embeddings In Kangas

3 min read

Visualizing Embeddings In Kangas

Source

Classification

Summary

Visualizing the embeddings in Kangas In this Jupyter Notebook, we construct a Kangas DataGrid containing the data and projections of the embeddings into 2 dimensions. What is Kangas? Kangas as an open source, mixed-media, dataframe-like tool for data scientists. It was developed by Comet, a company designed to help reduce the friction of moving models into p…

What This Teaches

  • How to connect OpenAI models with retrieval, embeddings, or external knowledge stores.

Implementation Use Cases

  • Use as a concrete implementation reference when building OpenAI API systems in this category.
  • Compare against current official API docs before copying model names, SDK calls, or parameters into production code.
  • Preserve this page as a mirrored source; prefer synthesis pages for personal recommendations or project-specific decisions.

Mirrored Content

Visualizing the embeddings in Kangas

In this Jupyter Notebook, we construct a Kangas DataGrid containing the data and projections of the embeddings into 2 dimensions.

What is Kangas?

Kangas as an open source, mixed-media, dataframe-like tool for data scientists. It was developed by Comet, a company designed to help reduce the friction of moving models into production.

1. Setup

To get started, we pip install kangas, and import it.

%pip install kangas --quiet
import kangas as kg

2. Constructing a Kangas DataGrid

We create a Kangas Datagrid with the original data and the embeddings. The data is composed of a rows of reviews, and the embeddings are composed of 1536 floating-point values. In this example, we get the data directly from github, in case you aren’t running this notebook inside OpenAI’s repo.

We use Kangas to read the CSV file into a DataGrid for further processing.

data = kg.read_csv("https://raw.githubusercontent.com/openai/openai-cookbook/main/examples/data/fine_food_reviews_with_embeddings_1k.csv")

We can review the fields of the CSV file:

data.info()

And get a glimpse of the first and last rows:

data

Now, we create a new DataGrid, converting the numbers into an Embedding:

import ast # to convert string of a list of numbers into a list of numbers
 
dg = kg.DataGrid(
    name="openai_embeddings",
    columns=data.get_columns(),
    converters={"Score": str},
)
for row in data:
    embedding = ast.literal_eval(row[8])
    row[8] = kg.Embedding(
        embedding,
        name=str(row[3]),
        text="%s - %.10s" % (row[3], row[4]),
        projection="umap",
    )
    dg.append(row)

The new DataGrid now has an Embedding column with proper datatype.

dg.info()

We simply save the datagrid, and we’re done.

dg.save()

3. Render 2D Projections

To render the data directly in the notebook, simply show it. Note that each row contains an embedding projection.

Scroll to far right to see embeddings projection per row.

The color of the point in projection space represents the Score.

dg.show()

Group by “Score” to see rows of each group.

dg.show(group="Score", sort="Score", rows=5, select="Score,embedding")

An example of this datagrid is hosted here: https://kangas.comet.com/?datagrid=/data/openai_embeddings.datagrid

Graph View

Backlinks