Apache Pinot
Original author(s) |
|
---|---|
Developer(s) | Apache Pinot |
Stable release | 1.0.0
/ 19 September 2023 |
Apache License 2.0 | |
Website | pinot |
Apache Pinot is a
Pinot was first created at
History
Pinot was started as an internal project at LinkedIn in 2013 to power a variety of user-facing and business-facing products. The first analytics product at LinkedIn to use Pinot was a redesign of the social networking site's feature that allows members to see who has viewed their profile in real-time. The project was open-sourced in June 2015 under an Apache 2.0 license and was donated to the Apache Software Foundation by LinkedIn in June 2019.[9][8]
Architecture
Pinot uses Apache Helix for cluster management. Helix is embedded as an agent within the different components and uses Apache ZooKeeper for coordination and maintaining the overall cluster state and health. All Pinot servers and brokers are managed by Helix. Helix is a generic cluster management framework to manage partitions and replicas in a distributed system.
Query management
Queries are received by brokers—which checks the request against the segment-to-server routing table—scattering the request between real-time and offline servers.
Cluster management
Pinot leverages Apache Helix for cluster management. Helix is a cluster management framework to manage replicated, partitioned resources in a distributed system. Helix uses Zookeeper to store cluster state and metadata.
Features
Pinot shares similar features with comparable OLAP datastores, such as
Pinot supports near real-time ingestion from streams such as
See also
References
- S2CID 12327343.
- ISBN 978-3-030-79382-1.
- ISBN 978-3-319-94301-5.
- S2CID 44083085.
- ^ "The Apache Software Foundation Announces Apache® Pinot™ as a Top-Level Project". blogs.apache.org. 2 August 2021.
- arXiv:2002.05839 [cs.CR].
- S2CID 21659844.
- ^ a b c Pawar, Neha. "Pinot Joins Apache Incubator" Archived 2019-04-02 at the Wayback Machine, LinkedIn Engineering, 01 April 2019
- ^ a b c Gopalakrishna, Kishore. "Open Sourcing Pinot: Scaling the Wall of Real-Time Analytics". engineering.linkedin.com. LinkedIn. Archived from the original on 10 September 2015. Retrieved 3 September 2020.
- ^ Yegulalp, Serdar (2015-06-11). "LinkedIn fills another SQL-on-Hadoop niche". InfoWorld.
- S2CID 232478317.
- ISBN 978-3-030-27520-4.
- ISBN 978-1-4920-7520-2.