 |
VOOZH | about |
|
VOOZH | about |
We’re so glad you’re here. You can expect all the best TNS content to arrive Monday through Friday to keep you on top of the news and at the top of your game.
Check your inbox for a confirmation email where you can adjust your preferences and even join additional groups.
Follow TNS on your favorite social media networks.
Become a TNS follower on LinkedIn.
Check out the latest featured and trending stories while you wait for your first TNS newsletter.
RALEIGH, N.C. — OpenSearch, a flexible open source data ingestion and analytics engine, was transferred in September by its creator, Amazon Web Services, to the Linux Foundation. In this On the Road episode of The New Stack Makers, Anandhi Bumstead, director of software engineering at AWS, talked to me about the project’s genesis and its journey to foundation sponsorship, and what comes next.
This episode of Makers was recorded at All Things Open in October.
The OpenSearch project began as a fork of Elasticsearch; in 2021, Elasticsearch’s open source Apache 2.0 license was changed to a more restrictive one. (In September, Elastic changed the licensing of Elasticsearch and its visualization dashboard, Kibana, introducing the open source GNU Affero General Public License as an option for users.)
For the OpenSearch project maintainers, the move to the Linux Foundation offered a number of advantages, Bumstead said.
“We really looked at a neutral foundation for neutral governance and also to bring in a broader community,” she said. “And if you look at foundations, like most foundations, they really facilitate neutral governance and also enabling companies to collaborate in open source.
“Linux Foundation has been very successful. I mean, tons of open source projects Linux Foundation has been governing. So we really wanted to lean into that.”
In his keynote address at the Open Source Summit in Vienna in September, Carl Meadows, director of product management for OpenSearch, likened the project to a “Swiss Army knife,” suitable for a variety of uses. Bumstead echoed that assessment.
OpenSearch, she told the Makers audience, “builds on the core search engine analytics, and also as a visualization out of the box.”
She added, “People widely use it for observability, log analytics. They use it for security analytics, for alert detection, and they’re heavily used, especially with the [generative AI] momentum, as a vector database. OpenSearch is also used for a lot of search scenarios. We have what is called semantic search, and we have this thing called hybrid search, where you can do a keyword search and you can do semantic search.”
These capabilities and the use cases they enable, she said, account for its reputation for flexibility.
However, OpenSearch has faced criticism from users for being slower than Elasticsearch in terms of indexing speed and handling complex queries.
“Earlier, we were focused on stabilizing and bringing the community on board,” Bumstead acknowledged. “But our primary focus continues to be performance, and we spent a lot of time tuning performance.”
Among the highlights of that “tuning,” she mentioned:
As for what’s next, she said, work continues on indexing, search and storage, and vector capabilities, “We’re doing a ton of work with performance and cost optimization in mind,” Bumstead added.
A big question the project maintainers are currently exploring, she said, is “How do we be more efficient in cost and storage? The thing which really excites me is some of the work we’re doing in that area. we’re doing work around [optimizing] storage in in the vector space. We would love to collaborate with more people to innovate in this area and really focus on vector performance and cost efficiency.”
Check out the full episode to learn more about OpenSearch and how it can be useful in conjunction with generative AI. Go to opensearch.org to contribute to the project.
