FAISS + SBERT实现的十亿级语义相似性搜索-FlyAI

FAISS + SBERT实现的十亿级语义相似性搜索

语义分割

最后更新 2020-11-09 14:51 阅读 4230

最后更新 2020-11-09 14:51

阅读 4230

语义分割

##FlyAI文献翻译
**英文原文：**[：Billion-scale semantic similarity search with FAISS+SBERT](https://towardsdatascience.com/billion-scale-semantic-similarity-search-with-faiss-sbert-c845614962e2)
> **标签** ：语义识别

![](https://static.leiphone.com/uploads/new/sns/article/202010/1603785351152144.png)

###Introduction

Semantic search is an information retrieval system that focuses on the meaning of the sentences rather than the conventional keyword matching. Even though there are many text embeddings that can be used for this purpose, scaling this up to build low latency APIs that can fetch data from a huge collection of data is something that is seldom discussed. In this article, I will discuss how we can implement a minimal semantic search engine using SOTA sentence embeddings ([sentence transformer](https://arxiv.org/pdf/1908.10084.pdf "sentence transformer")) and [FAISS](https://github.com/facebookresearch/faiss "FAISS").

>###介绍

>语义搜索是一种关注句子意义而不是传统的关键词匹配的信息检索系统。尽管有许多文本嵌入可用于此目的，但将其扩展到构建低延迟api以从大量数据集合中获取数据是很少讨论的。在本文中，我将讨论如何使用SOTA语句嵌入（语句转换器）和FAISS来实现最小语义搜索引擎。

###Sentence transformers

It is a framework or set of models that give dense vector representations of sentences or paragraphs. These models are transformer networks(BERT, RoBERTa, etc.) which are fine-tuned specifically for the task of Semantic textual similarity as the BERT doesn’t perform well out of the box for these tasks. Given below is the performance of different models in the STS benchmark

>###句子Transformers

>它是一个框架或一组模型，给出句子或段落的密集向量表示。这些模型是transformer网络（BERT、RoBERTa等），它们专门针对语义文本相似性的任务进行了微调，因为BERT在这些任务中执行得不是很好。下面给出了不同模型在STS基准测试中的性能。

![](https://static.leiphone.com/uploads/new/sns/article/202010/1603785351734285.png)

Image source: Sentence transformers

We can see that the Sentence transformer models outperform the other models by a large margin.

But if you look at the leaderboard by papers with code and GLUE, you would see many models above 90. So why do we need Sentence transformers?.

Well, In those models, the semantic Textual similarity is considered as a regression task. This means whenever we need to calculate the similarity score between two sentences, we need to pass them together into the model and the model outputs the numerical score between them. While this works well for the benchmarking test, it scales badly for a real-life use case, and here are the reasons.

>图片来源：句子 transformers

>我们可以看到句子transformer模型比其他模型有很大的优势。

>但是如果你用代码和GLUE来看看排行榜，你会看到很多的模型超过90。为什么我们需要句子transformers？

>在这些模型中，语义文本相似度被视为一个回归任务。这意味着，每当我们需要计算两个句子之间的相似度得分时，我们需要将它们一起传递到模型中，然后模型输出它们之间的数值分数。虽然这对于基准测试很有效，但是对于实际的用例来说，它的伸缩性很差，原因如下。

>1.当你需要搜索大约10k个文档时，你需要进行10k个独立的推理计算，不可能单独计算嵌入量而只计算余弦相似度。见作者的解释。

>2.最大序列长度（模型一次可以接受的单词/标记的总数）在两个文档之间共享，这会导致的表示的含义由于分块而被稀释

###FAISS

Faiss is a C++ based library built by Facebook AI with a complete wrapper in python, to index vectorized data and to perform efficient searches on them. Faiss offers different indexes based on the following factors

- search time

- search quality

- memory used per index vector

- training time

- need for external data for unsupervised training

So choosing the right index will be a trade-off between these factors.

>###FAISS

>Faiss是一个基于C++的库，由FacebookAI构建，在Python中有完整的包装器，用于索引矢量化数据并对其进行有效的搜索。Faiss基于以下因素提供了不同的索引。

> - 搜索时间

>- 搜索质量

>- 每个索引向量使用的内存

>- 训练时间

>- 无监训练需要外部数据

>因此，选择合适的指数将是这些因素之间的权衡。

###Loading the model and performing the inference on the dataset

First, let us install and load the required libraries

>###加载模型并对数据集执行推理

>首先，让我们安装并加载所需的库

```
!pip install faiss-cpu
!pip install -U sentence-transformersimport numpy as np
import torch
import os
import pandas as pd
import faiss
import time
from sentence_transformers import SentenceTransformer
```
Loading the dataset with a million datapoints

I used a dataset from Kaggle that contains news headlines published over a period of seventeen years.

>加载一个包含一百万个数据点的数据集

>我使用了一个来自Kaggle的数据集，其中包含了17年来出版的新闻标题。

```
df=pd.read_csv("abcnews-date-text.csv")
data=df.headline_text.to_list()
```
Loading the pre-trained model and performing the inference

>加载预训练模型并且进行推断

```
model = SentenceTransformer('distilbert-base-nli-mean-tokens')encoded_data = model.encode(data)
```
###Indexing the dataset

We can choose different indexing options based on our use case by referring to the [guide](https://github.com/facebookresearch/faiss/wiki/Guidelines-to-choose-an-index "guide").

Let's define the index and add data to it

>###为数据集编制索引

>我们可以根据我们的用例通过参考指南来选择不同的索引选项。

>让我们定义索引并向其添加数据

```
index = faiss.IndexIDMap(faiss.IndexFlatIP(768))index.add_with_ids(encoded_data, np.array(range(0, len(data))))
```
Serializing the index
>序列化索引

```
faiss.write_index(index, 'abc_news')
```
The serialized index can be then exported into any machine for hosting the search engine

Deserializing the index

>将序列化的索引导出到托管搜索引擎的任何计算机中

>反序列化索引

```
index = faiss.read_index('abc_news')
```
###Performing the semantic similarity search

Let us first build a wrapper function for search

>###执行语义相似性搜索

>让我们首先为搜索构建一个包装函数

```
def search(query):
   t=time.time()
   query_vector = model.encode([query])
   k = 5
   top_k = index.search(query_vector, k)
   print('totaltime: {}'.format(time.time()-t))
   return [data[_id] for _id in top_k[1].tolist()[0]]
   ```
performing the search
>执行搜索

```
query=str(input())
results=search(query)
print('results :')
for result in results:
   print('\t',result)
   ```
   
###Results on CPU

Now let's take a look at the search results and response time

>###CPU中的结果

>现在让我们看看搜索结果和响应时间

![](https://static.leiphone.com/uploads/new/sns/article/202010/1603785353408133.png)

1.5 seconds is all it takes to perform an intelligent meaning-based search on a dataset of million text documents with just the CPU backend.

>只需1.5秒，就可以在仅使用CPU后端的百万文本文档的数据集上执行基于意义的智能搜索。

###Results on GPU

First, let's uninstall the CPU version of Faiss and reinstall the GPU version

>###GPU中的结果

>首先让我们关闭CPU版本的Faiss并重启GPU版本

```
!pip uninstall faiss-cpu
!pip install faiss-gpu
```
Then follow the same procedure, but at the end move the index to GPU.

>之后执行相同步骤，但是最后将索引移到GPU上.

```
res = faiss.StandardGpuResources()
gpu_index = faiss.index_cpu_to_gpu(res, 0, index)
```
Now let's place this inside the search function and perform the search with the GPU.

>现在让我们转移这个搜索方法并用GPU执行这个搜索

![](https://static.leiphone.com/uploads/new/sns/article/202010/1603785355933744.png)

That’s right, you can get the results within 0.02 sec with a GPU ( Tesla T4 is used in this experiment) which is 75 times faster than a CPU backend

>很好，你可以在0.02秒内得到结果，使用GPU（在这个实验中使用了Tesla T4），它比CPU后端快75倍

**But why can’t I just serialize the NumPy array of the encoded data instead of indexing them and use cosine similarity if I can wait for a few seconds?**

Because the NumPy doesn’t come with native serialization functions, hence the only way is to convert it into a JSON and then save the JSON object, but then the size will increase by a factor of five. For example, a million data points encoded into a 768-dimensional vector space with normal indexing will be about 3GB, converting it into JSON will make it 15GB, which a normal machine can’t hold on it’s RAM. So we will have to run a million computational inference each time we perform a search, which is not practical.

>**但是为什么我不能仅仅序列化编码数据的NumPy数组而不是索引它们呢？如果我能等几秒钟的话，使用余弦相似性呢？**

>因为NumPy没有序列化函数，因此唯一的方法是将其转换为JSON，然后保存JSON对象，但是大小将增加五倍。例如，在768维向量空间中编码的一百万个数据点具有正常的索引，大约为3GB，将其转换为JSON将使其成为15GB，而普通机器无法保存它的RAM。因此，每次执行搜索时，我们都要运行一百万次计算推理，这是不实际的。

###Final Thoughts

This is a basic implementation and a lot needs to be still done both on the language model part and the indexing part. There are different indexing options from which the right one should be chosen based on the use case, size of the data and the compute power available. Also, the sentence embedding used here is just fine-tuned on some public datasets, fine-tuning them on the domain-specific dataset would improve the embeddings and hence the search results.

>###最后的想法

>这是一个基本的实现，在语言模型部分和索引部分仍然需要做很多工作。有不同的索引选项，应该根据用例、数据大小和可用的计算能力选择正确的索引选项。另外，这里使用的句子嵌入只是对一些公共数据集进行了微调，在特定领域的数据集上对它们进行微调可以改进，从而提高搜索结果。

References

>参考文献
[1] Nils Reimers and Iryna Gurevych. “[Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation.](https://arxiv.org/pdf/2004.09813.pdf "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation.")” arXiv (2020): 2004.09813.

>[2]Johnson, Jeff and Douze, Matthijs and J{\’e}gou, Herv{\’e}. “[Billion-scale similarity search with GPUs](https://arxiv.org/abs/1702.08734 "Billion-scale similarity search with GPUs")” arXiv preprint arXiv:1702.08734.

本作品采用知识共享署名-非商业性使用-相同方式共享 4.0 国际许可协议进行许可，转载请附上原文出处链接和本声明。
本文链接地址：https://www.flyai.com/article/751

讨论

500字

表情

发送

删除确认

是否删除该条评论？

取消删除