In-depth: PostHog vs GrowthBook

Oct 10, 2023

PostHog and GrowthBook both provide open source, self-serve feature flags and experimentation, but they're different in two important ways:

In this post, we'll cover these differences in more detail, and answer frequently asked questions about both tools.

How is PostHog different?

1. PostHog is an all-in-one platform

PostHog brings together all the tools engineers need for testing, releasing, and measuring the success of new features. Feature flags and A/B testing are only part of a suite of tools PostHog offers.

PostHog homepage features

PostHog combines usage, performance, and behavioral data with flags and experiments. PostHog’s data warehouse also enables you to pull in data from external sources.

Having all these product and data tools together enables you to do better analysis of shipped features and make better decisions about what you are building next.

2. PostHog is built for startups and engineers

PostHog is built for high-growth startups. This means it is simple for founders and engineers to implement themselves. There are many SDKs, tutorials, and docs to help you get started quickly with any type of app.

As startups scale, PostHog also provides the more advanced tools they need to succeed. These include advanced product analytics, SQL querying, CDPs, and data warehousing.

GrowthBook, on the other hand, focuses only on later-stage, larger companies than PostHog. Many of their features, like their analytical A/B testing suite, are great for data teams which also come at a later stage.

3. PostHog is easier to setup

GrowthBook requires more setup than PostHog as it relies heavily on external tools and writing SQL:

  1. To get data into GrowthBook, you must integrate a data source.
  2. To track data related to flags and experiments, you connect to an analytics tool.
  3. To configure data sources and track events, you write SQL configurations in GrowthBook.
  4. To set up goals and analysis, you write more SQL.

PostHog needs none of this. You use the same app and SDKs for managing and evaluating feature flags as tracking usage. There is no extra configuration needed. Creating flags, experiments, and insights doesn’t require writing SQL. This makes PostHog faster to get started and easier to use once set up.

Comparing PostHog and GrowthBook

Platform

Both PostHog and GrowthBook have the infrastructure to use flags and experiments effectively with your current app. PostHog has a wider range of product and data tools built in, while GrowthBook relies on third-party integrations

PostHogGrowthBook

Open source

Code publicly accessible

Self-service

No need to talk to sales

SDKs

Including JavaScript, Android, iOS, and Python

1411

Native data sources

Compute metrics and results without integrations

Proxies

Reverse proxy to avoid blockers

Self-hostable

Host on your own infrastructure

API

Edit and evaluate flags with API

Beta

Local evaluation (aka streaming)

Faster flags by not having to rely on the server for evaluation

SQL

Query flag and product data directly via SQL

Session replays

Real user playback of flag session

Feature flags

Both PostHog and GrowthBook offer all the functionality you expect from feature flags.

PostHogGrowthBook

Boolean flags

Simple flags returning true or flag

Percentage rollouts

Target percentages of a group

Custom targeting

Target users based on user properties, custom contexts

JSON payloads

Flags return JSON

Environments

Manage flags for dev, staging, prod

Partial

Scheduling

Schedule flags to turn on or off

Early access management

Manage betas, test features

Bootstrapping

Flags available on frontend application load

  • Targeting: GrowthBook's custom targeting using attributes must be set every session and defined in-app before use. It's also only available on the paid Pro plan. PostHog automatically sets its equivalent (properties) on users. You don’t need to pre-define them and unlimited custom values are free.

  • Bootstrapping: PostHog’s JavaScript web SDK enables you to pass flags directly from the backend before the app loads. This ensures they're available immediately and prevents flickering. GrowthBook instead recommends moving the A/B test or flag logic earlier in the page load (server-side) to prevent this.

Experimentation

Experimentation is where PostHog and GrowthBook’s functionality diverges. Both enable you to create A/B/n tests with custom goals and calculate statistical significance, but:

  • GrowthBook’s A/B testing provides more potential statistical depth, such as Bayesian and Frequentist engines, value capping, regression adjustment, and testing corrections. These do require statistics expertise to use reliably, though.

  • PostHog provides a simpler interface and no need to write SQL or connect to external services – a better experience for users without a background in data science.

PostHogGrowthBook

Custom goals

Customize metrics that test tracks

Secondary metrics

Monitor impact on unrelated metrics

Native goal tracking

Track impact on goals without external sources

Statistical significance calculation

Calculate if changes make a statistically significant impact

Split testing

Split participants into groups

Multivariate (A/B/n) testing

Test multiple variants of a change

Recommended run time

Calculate the recommended run time for your experiments

Namespaces

Avoid conflicts by assigning all users to a namespace

Visual editor

Create and edit experiments in your app

Statistics engine

How the success of an experiment is calculated

BayesianBayesian, Frequentist
  • Recommended run time: PostHog automatically calculates a recommended run time based on past data and minimally acceptable improvements. This helps you avoid the peeking problem and end your experiment at the right time.

Recommended run time calculator

  • Visual editor: GrowthBook’s visual editor is similar to PostHog’s toolbar. It enables you to select an element to modify using an A/B test. The editor is limited to server-side rendered apps.

  • Frequentist engine: GrowthBook has an additional statistical analysis option. It enables you to compare the control with the experiment using p-values and confidence intervals.

Reporting and analytics

GrowthBook offers no reporting or analytics tools other than the ability to query experiment reports with SQL. As a result, GrowthBook forces users to rely on external event tracking and visualization tools.

PostHog provides all the visualizations and product tools for evaluating the success of your features and app. These include trends, funnels, retention, SQL querying, and session replays integrated with your flags and A/B testing data. This enables you to do deeper analysis of their impact and combine them with other product and usage data.

PostHogGrowthBook

Analytics

Capture usage and calls related to flags

Experiment reports

Show the results of an experiment

Trends

Graphs of flag usage

Funnel visualization

Analyze impact of flags on funnels

Retention

User retention related to flags

Breakdowns

Breakdown reports by properties

Dashboards

Multiple customizable insights for a flag

Non-flag data

Combine flag data with other product data

Pricing

PostHogGrowthBook

Pricing

Per requestSeat-based

Free plan

Free experiments

Free collaboration

3 seats free, then $20/seat/month

Fully transparent

PostHog’s feature flag pricing is pay-per-request (and A/B tests use feature flags). There is a generous free tier of 1M requests per month with all features, add-ons, and integrations available.

Pricing

Like PostHog, GrowthBook is free to self-host. GrowthBook Cloud is a seat-based model. It is free for up to 3 users, then $20/user/month. They claim this provides "unlimited traffic," but in reality, the limit is 10M requests per month, after which you need an enterprise plan.

Growthbook

Features, like flag scheduling, permissions, custom fields, and the visual editor are only available on the GrowthBook Pro paid plan.

Example scenarios

To give you an idea of what pricing looks like in reality, here are some example situations and their estimated costs for both PostHog and GrowthBook.

Note: GrowthBook does not display their Enterprise pricing needed to go beyond 10M requests per month.

SeatsRequestsPostHog costGrowthBook cost
31,000,000$0$0
52,000,000$100$100
154,000,000$300$300
2015,000,000$1400??? (Enterprise)

Notes:

  • Using backend local evaluation in PostHog lowers the amount of flag usage depending on the polling duration and active number of servers. If you use locally evaluated flags with one server polling every 30 seconds, this amount is under 1M requests (free).
  • PostHog has volume discounts on flags over 2 million requests per month.

Integrations

GrowthBook has integrations with data warehouses and analytics tools they rely on, but little beyond that. PostHog has a wider range of integrations.

PostHogGrowthBook

Exports

Export data to other sources

Imports

Import data from sources

Slack

Alerts for Slack

MS Teams

Alerts for Microsoft Teams

Zapier

Trigger Zapier automations

Sentry

Connect to Sentry data

Datadog

Capture flag data in Datadog

PostHog’s event-based structure enables you to import data from anywhere for use with flags and experiments. The free API enables you to connect, edit, and capture from anywhere too. For example, you can import data from warehouses, no-code site builders, Segment, and more.

Security and compliance

Both PostHog and GrowthBook enable companies to remain secure and compliant with privacy regulations. Companies can customize the levels of user privacy related to these platforms to their needs.

PostHogGrowthBook

User privacy options

Anonymize users, drop personal data

History, audit logs

Manage and view flag edits and related users

Enterprise

GDPR-ready

Can be compliant with GDPR

HIPAA-ready

Can be compliant with HIPAA

SOC 2

SOC 2 security certification

2FA

Enforce login with two-factor authentication

SAML/SSO

Use SAML or single sign-on authentication

EnterpriseEnterprise

Frequently asked questions

Who is PostHog useful for?

PostHog is built for startups and their engineers. It provides all the tools startups need to build successful products. The people who find PostHog most useful are founders, product engineers, and growth engineers.

Companies that use PostHog feature flags and experiments include Y Combinator, Vendasta, and AssemblyAI.

Who is GrowthBook useful for?

Larger organizations that want their feature flag and experimentation tools use GrowthBook. They often already set up data warehouses and analytics tools. GrowthBook is an added piece to their toolkit. In these organizations, it is used by a combination of engineering, data, and product teams.

Companies that use GrowthBook include Patreon, Deezer, and Pepsi.

How much does PostHog cost?

Feature flags and experiments are free for up to 1M requests per month. Beyond that, it costs $0.0001/request (or $1 per 10,000 requests). There are discounts for high-volume users, non-profits, and startups.

Other products, like product analytics and session replay, have separate but similarly structured pricing.

How much does GrowthBook cost?

GrowthBook, like PostHog, is free to self-host. This doesn’t count infrastructure, implementation, or maintenance costs.

GrowthBook Cloud is free for up to 3 users and 10M requests per month. Its "Pro" plan is $20/seat/month and provides access to premium features. They also have an Enterprise plan that adds more security and compliance tools as well as a higher request limit.

Do GrowthBook or PostHog offer free trials?

GrowthBook is free to self-host and has a free cloud tier with unlimited flags and experiments for up to 3 users.

PostHog lets you use all its features for free. Once you upgrade to paid, you get 1M flag requests, 1M events, and 5,000 recordings for free each month. If your usage stays below this, PostHog remains free.

How long does it take to implement PostHog?

Feature flags and experiments are simple. They are a few lines of code in any of your favorite languages. They can even be set up on no-code site builders like Framer or Webflow.

The process requires signing up for PostHog, installing the snippet or SDK in your app, creating the flag in PostHog, and implementing the flag evaluation and relevant logic in your app. Many of the SDKs handle important aspects like local evaluation and event capture for you.

You can reuse the PostHog implementation, like user identification, across products. Because PostHog is an all-in-one platform, analytics capture for targeting and A/B testing results doesn’t need set up or connection either.

How long does it take to implement GrowthBook?

The basic GrowthBook implementation seems relatively similar to PostHog, but the need to define data twice as well as integrate external tools makes it longer.

To implement GrowthBook from scratch:

  • Sign up or self-host.
  • Create an SDK endpoint, install the SDK in your app, and initialize the SDK.
  • Set up target attributes in-app and define them in GrowthBook.
  • Set up an analytics tool to track events and integrate it with GrowthBook.
  • Connect your data source (warehouse) and write SQL to configure it.
  • Create a feature flag or experiment. Set up tracking metric(s) using SQL. Implement related logic in-app.

Although GrowthBook provides templates for integrating external tools, this whole process takes longer than PostHog.


Ready to find out more?


Comments