Accelerating Open Source growth: the ever expanding network effect? 3 key takeaways from fast growing companies and inputs from their founders

Open source has been around for a while, however the scale of traction and adoption of some of the leaders has been so massive it was almost surprising for customers, the community and closed source software providers.

Indeed, some of the open source leaders of verticals were able to grab massive adoption — and high valuations. Here below some of these famous open source leaders and traction achieved (public information, Q1 2022)

Is this trend towards massive growth of open source going to accelerate? With a world adopting new generations of technologies faster and faster, and community adoption and network effects that grow fast and faster, one may wonder if open source may become a much bigger channel in the future, winning against competing private closed source software. There are however also many open source companies that do not make it to stellar community adoption or high valuations. Actually, some of these open source companies do not need or want to have commercial offerings — for some their purpose is a community one.

Open source companies may address similar problems as closed source software — differently. The type of users and angle taken often allows coexistence of open and closed source software on the market. A same problem can often be addressed in different ways that correspond to different customer profiles and desires. For instance customers sensitive to a lower pricing, transparency, product led growth and community involvement may favor open source. While others that are more comfortable with a product with closed end source code and do not require community involvement in the production of a product and its customizations will prefer to stick to closed end products. Hence one may believe that markets will continue welcoming both open and closed source software.

But for those open source companies that want to one day to have commercial offerings with massive adoption, is there a secret recipe? The founders of four top open source companies were asked about their thoughts on the challenges, opportunities and how they coordinated community growth to build their product and awareness. It appears that:

Four fast growth open source companies were asked about the challenges and opportunities with open source

Why does open source help and how were these companies successful in growing community adoption and traction with open source? Four companies below have been successful in getting fast open source adoption. They address complementary problems. They and their founders were kind enough to share some insights:

Now into more details for each:

• Origins and problem solved: Airbyte was founded in 2020, to address data integration through open source. Data connectors require constant updates and are expensive to maintain. This is a developer first problem, and one where transparency and a lower pricing lends itself well to open source approach.
• How it works and where open source helps: Data integration lends itself well to open source. There are many closed source providers that offer data connectors, however covering the long tail of connectors comes with a price for customers. Airbyte offers to address the long tail with a community-led maintenance and the ability to adapt and customize these connectors to every user needs. Customers may expect cheaper, transparent, broader and resilient data connectors. When this is tackled, companies can finally focus on insights and innovation.
• Strategy used, community growth and fundraising: from its beginning, Airbyte structured itself for open-source community-led adoption: 1) With a fully transparent culture and values: everything is available on the website, from the company values to its roadmap, logo choice, documentation, all the way up to visibility into connector connectivity. (resources: https://handbook.airbyte.com/ https://airbyte.com/contributors) 2) The company aligned processes and rewards for community activism, which led to its strong and sticky growth. For instance Airbyte carefully aligns user success to user goals with rewards and personally onboards every new channel joiner (https://airbyte.com/community) 3) From a product point of view, Airbyte guarantees a very quick time to resolution (the average is as low as 2.5 hours). The team makes it a priority to close every question fast. The community helps. 4) Finally, Airbyte innovated on the business model with pricing that does not necessarily rely on volume of data only, leading to an overall much lower price point than competition.

With all this, Airbyte has achieved since 2020, hence in less than a year, 10K users, 5K stars on Github, 5K members on Slack and 2.2K followers on Linkedin. Airbyte raised +$180M with investors including Accel, Benchmark, Coatue, Altimeter, Salesforce and Thrive, with the latest round of $150M at $1.5Bn valuation in late 2021.

• Origins and problem solved: dbt Labs was founded in 2016 and was originally a non-commercial initiative by consultancy Fishtown Analytics. dbt is a development framework that combines modular SQL with software engineering best practices to make data transformation reliable, fast, and fun. Strong adoption combined with the open core means that all dbt cloud customers also use dbt Core (the open source product). Up to 8k companies are now said to use dbt. dbt is becoming one of the standards in data transformation, making high volumes of data in cloud data warehouses usable for visualization/BI. In doing so, dbt helps analytics engineers transform, test, deploy and document data. In an organization’s landscape of cloud data warehouses, databases and data ingestion/pipelining, dbt helps transform data through SQL scripts within the data warehouse vs. outside.
• How it works and where open source helps: dbt is a command line tool enables data analysts to write dbt code in a simple text editor, then call this code with a command line. dbt will then compile this into SQL and push it to database. dbt cloud with then serve as a hosted service to help data analysts with the production of dbt deployments. Reliance on SQL is an important part as it simplifies data analysts/engineers’ life and enables them to work faster without having to worry about dependencies. Open source has been a key element drive and feed product changes from community feedback.
• Strategy used, community growth and fundraising: dbt developed a precise open source strategy by: 1) Promoting awareness by encouraging the community to talk about their structural data problems at social gatherings / meetups; basically encouraging sharing and collaboration in the community. (https://www.getdbt.com/community/) 2) Closing on a few select and well-known local customers 3) Requesting community feedback on switch from on prem to cloud and challenges

Since 2016, dbt has reached 4K stars on Github, 22K members on Slack and ~10K followers on Linkedin and +1K dbt customers. Over the years, dbt has attracted +$190M in funding from Sequoia, Andreessen Horowitz, Amplify and Altimeter, with the latest round in 2021 of $150M at $1.5Bn valuation.

Dbt Labs data growth in 2021 (source: dbt, Q2 2021)

• Origins and problem solved: Jina was founded in 2020, as the neural search company, to tackle information finding through deep learning neural networks, especially with unstructured data (images, videos etc.). This is a problem as search is still algorithm/criterion based and neural network applications may extend scope and possibilities to search for nearly everything without criteria. Other companies rely on a simple to relatively complex algorithm or attempt to tackle this issue through community/open source led participation to improve these algorithms, which also entail tackling issues of volume data, indexing, etc. Neural search applications are elegant in that they theoretically solve for the issue in infinite manner.
• How it works and where open source helps: With Jina, companies go beyond keywords and into endless possibilities and ROI towards faster, more effective search, in a process that basically turns data bits collected into a series of mathematic distance vectors, which is then used in the search algorithms, thereby solving for relevance, accuracy, data refresh and efficiency. Overall, several approaches exist to search and Jina brings one that should solve for any type of search, and especially for unstructured elements in a manner that can theoretically cover an infinity of applications/volume. Other companies may provide great search experience too, maybe more catered to other types of applications/purposes.
• Strategy used, community growth and fundraising: Jina focused on: 1) Product led growth with development of the product with the community to create a perfect product and making sure it is accessible to use. Open-sourcing the technology helped the growth and adoption of Jina in a massive scale and speed to validate the value of the product 2) Development and gathering varied use cases and with fast feedback loops 3) Making use of the community to generate more data and data environments (https://jina.ai/join/)

Since its creation in 2020, Jina raised +$35m from investors such as Canaan, Mango, GGV, SAP and Yunqi Partners among others. The Jina developer community has +1K users, and many applications (video games, legal, corporate, etc.), 13K stars on Github and 5.5K followers on Linkedin.

Jina offering (source: Jina, Q4 2021)

• Origins and problem solved: Origins and problem solved: Prefect was born in 2018 as a dataflow automation tool that enables orchestration of distributed data pipelines. The company’s mission is to “eliminate negative engineering:” the necessary but frustrating and time-consuming effort to prepare code for unexpected errors and failures. Negative engineering problems most frequently emerge whenever code or data transitions between two domains: from data scientists to engineers; from dev to production; from database to data warehouse. Each handoff increases the risk of error while reducing overall visibility and lineage. The company distributes Prefect, an open-source workflow management system, and Prefect Cloud, a managed orchestration platform. The two products work together via a Hybrid Execution Model that keeps scheduling and orchestration activity in the cloud while ensuring that users’ data and code stays secure and private on-premise. One of Prefect’s core principles is to meet users where they are and let them continue using their preferred tools without changing their workflow, even as they benefit from the orchestrator’s enhancements.
• How it works and where open source helps: While other software requires users to generate static config files or write orchestrator-specific code, Prefect embraces “code as workflows” and makes it as easy as possible to automatically enhance existing code with orchestration benefits like distributed pipelines, observability, and custom infrastructure. Companies love it because they can test and run workflows quickly, with minimal changes and on their own infrastructure. Thanks to the hybrid model, running those same workflows against the Prefect Cloud service is just a toggle away. Furthermore, the open-source community is extremely active and one of the fastest growing in the data space, making Prefect increasingly at the center of modern data stack activity.
• Strategy used, community growth and fundraising: Prefect: 1) Relied on the open source community to reach many use cases from stakeholders working in very different industries (FSI, space, etc.) to improve the product. 2) Offered full transparency in the product development and to adapt the product to be compatible with any code and structure so that end users don’t have to adapt to anything. 3) Developed functionalities requested by the community: automatic scheduling, permission, remote user clusters, etc (https://www.prefect.io/community/updates/)

Since launching publicly in 2019, Prefect has reached 13k Slack community members, 8k GitHub stars, 8k Twitter followers, and 5K LinkedIn followers. Prefect Cloud processed more than 500M tasks last year and is used by some of the largest companies in the world, including Cisco, Capital One, Progressive Insurance, and Block. They’ve raised over +$57M with investors including Bessemer and Tiger.

Prefect data flow automation (source: Prefect, Q1 2022)

So what were some key takeaways to accelerate open source growth?

Open source growth may seem enigmatic, difficult to predict and anticipate. Yet, three takeaways seem to emerge from these success stories, which may increase adoption and open source growth:

1. While it may seem difficult to predict community adoption, these fast growing open source companies developed a coordinated and precise strategy to achieve it. Open source relies on community adoption. But communities can be volatile, and adoption difficult to predict and many find it difficult to predict which open source company will achieve community adoption. However, the CEOs of some of the top performing companies mentioned how they took a very structured approach to building the communities. From communications, to transparent everything, to community perks — it seems there was no chance involved in taking off. Like social networks and social stars, those who make it actually have trained and strategized to make it. Best is to check out the communities to know more: https://airbyte.com/community, https://www.getdbt.com/community/, https://jina.ai/join/, https://www.prefect.io/community/updates/.

2. While continuous product improvements may seem challenging with open source — since it relies on many constituents — an approach that was process-driven, rewarding and with personalized channel approached worked. One recurring piece of skepticism against open source is the difficulty in harnessing productivity towards product improvement. In the course of this article creation, an interviewed customer spoke about the ‘idealistic developer’ who wants changes and improvements in existing software and, tired of waiting for these, resorts to open source as an alternative. The detractors here mention the difficulty in coordinating a community towards structure software construction. However, founders of top performing open source companies mentioned that the main benefits of open source is that products are continuously worked on and improved by the community. The ability to funnel this network-like collaboration can enable much greater productivity. Through a super structured continuous improvement approach, great coordination and a personalized approach to welcoming and engaging with the community, successful open source companies are able to act on queries, gather and improve on feedback and develop products much better. Usage of the right channels and tools (Github, Slack, LinkedIn, etc.) is key, as are community rewards.

3. While the initially free business model may seem counterintuitive with regards to commercial success, strategized adoption leading to better utilization and lower price point was successful when done properly. Open source is all about an initial free component. The community adopts, then additional premium services are sold on top, usually around custom build components, or added layers of security, identification, hosting, etc. Enterprises are ready to pay significant amounts for these. The timing of the monetization is a question– wait for fully fledged community growth, or start testing monetization options sooner? With the world increasingly becoming deflationary in nature for certain tech areas (mass consumption of software goods leads to a decrease in price), community adoption, lends itself very well to lower pricing. But open source is a great way to take this relationship the other way around — first lead with community adoption, then create community adoption share, and finally conquer actual paid market share by providing lower priced but equally good quality software for those who want these. A good business model from the outset, that is aligned to the community can help the community broadly. This initially enables any developer to take advantage of amazing technologies without paying an arm and leg for it. So it serves best all users. The open source business model is not free forever for those that want customized and added premium layers, it is strategized to start with community adoption and end with monetized and often lower priced software for the benefit of both the community and companies that choose the premium options they need on top. This is sometimes better aligned with the current world’s consumption model — lower price for mass consumption and premium options/prices for those that need premium features.

Overall, both open and close source software will likely continue to see great growth ahead. Open source especially will likely continue seeing bright days as the network effects and community involvement become faster and more prevalent. Some customers (developers and companies) will always prefer close source for the ability to resort of a single company with coordinated management for instance. Other customers will favor open source for the transparency, community involvement leading to a product led growth and often lower pricing — at least initially and depending on premium features then added on case by case basis. For those open source companies wishing to reach high growth and fast community adoption (not all do, and some do not lend themselves to commercial offerings), the world has never been more welcoming: communities grow fast and network effects are large. But it requires carefully laid out strategies, as is evidenced by the four open source companies described, Airbyte, dbt, Jina and Prefect.

Three elements stood out: 1) a coordinated and precise strategy to achieve community growth; 2) continuous product improvements enabled by structured processes, rewards and personalized channel management and 3) an initially free or lower pricing business model leading to adoption and utilization with the activation then of key premium layers (hosting, privacy, security, etc.) and customizations that allow monetization.

Open source, and both open and closed source software seem to have bright days ahead. The world has never been more welcoming of great software with network effects, carefully developed to address community and/or individual company needs. For those that success, growth will likley happen faster and faster!

Special thanks to all contributors (big thanks to Sakib Dadi at Bessemer, Will Sheldon at Accel, Megan Reynolds at Crane, etc.), including the four companies and founding teams but also other the select investors, customers, partners and other open-source entrepreneurs interviewed.