-
A new proprietary vector-search-oriented database, built statelessly on object storage (S3) with “smart caching” on SSD/RAM — “a solution that scales effortlessly to billions of vectors and millions of tenants/namespaces”. Apparently it uses a new storage engine: “an object-storage-first storage engine where object storage is the source of truth (LSM). […] In order to optimize cold latency, the storage engine carefully handles roundtrips to object storage. The query planner and storage engine have to work in concert to strike a delicate balance between downloading more data per roundtrip, and doing multiple roundtrips (P90 to object storage is around 250ms for <1MB). For example, for a vector search query, we aim to limit it to a maximum of three roundtrips for sub-second cold latency.” HN comments thread: https://news.ycombinator.com/item?id=40916786
(tags: aws s3 storage search vectors vector-search fuzzy-search lsm databases via:hn)