Checking access…

Built on Google Cloud
Omni Enterprise

Enterprise
Growth Intelligence

Network-native growth intelligence for enterprise app verticals — Ride Hailing, Food Delivery, Shopping, Banking, Insurance, AI tools, and more. Built on real operator engagement signals, Omni Enterprise tracks how categories grow, how brands compete within them, and where the opportunities are — delivered as an interactive dashboard platform and structured reports, refreshed monthly.

~82M Subscribers 25 Dashboard Views 8 Core Datasets S3 + Dashboard Delivery
Omni Enterprise — Enterprise Growth Intelligence
Product Overview

Network-Native Growth Intelligence for Mobile App Categories

Omni Growth is a behavioural intelligence platform built entirely on live network observability data — not surveys, not declared panels. It tracks how mobile app categories and individual brands grow, compete, and reach their audiences across the operator's subscriber base, and frames every insight inside a rigorous Byron Sharp growth framework.

What Omni Growth Is

Every metric in the platform traces back to a single source: real engagement signals observed on the operator's network infrastructure. When a subscriber opens an app, makes a payment, or sustains an hour-long session, that event is captured — deduplicated, classified, and aggregated into monthly snapshots across eight structured datasets.

The result is a category-first view of the mobile app economy. Rather than starting from a brand's own analytics, Omni Growth starts from the full network panel — all observed mobile subscribers — and measures how every brand sits within it. This means penetration rates, intensity distributions, competitive overlaps, and social channel reach are all expressed against a common, live denominator rather than against self-reported or estimated bases.

The platform is grounded in Byron Sharp's How Brands Grow framework. The Double Jeopardy Law — that smaller brands have both fewer users and lower loyalty — is directly observable here. Share of Requirements, the proportion of a category's total sessions captured by each brand, is a first-class metric. Healthy growth is defined as a rising Light-user share: broad, shallow reach is more durable than deepening engagement among a narrow Heavy base.

Panel Size
~82M
Network mobile subscribers observed
Geography
126
Regional geographic districts
Session Types
6
open · login · commerce · more
Dashboard Views
25
across 5 strategic question groups
Refresh
Monthly
M+5 business day availability

Five Questions, One Platform

The 25 dashboard views are organised around five strategic questions. Each question has a dedicated view group, and the underlying datasets are the same across all of them — so answers to one question inform the others.

01
How is the category growing?

Tracks overall category health across six months: total active users, session volumes, panel penetration %, and the Light / Medium / Heavy intensity split that reveals whether growth is broad or concentrated. Sliced by age band, gender, geography (top regional areas), and engagement intensity so you can see exactly which demographic or regional pocket is driving momentum.

Panel SizeCategory BehaviourTrends by Age · Gender · Geography · IntensityCategory Penetration
02
How is your brand growing?

Positions a brand within its category using the Double Jeopardy Law — brands with higher penetration will always have higher loyalty too, so the question is how fast you are moving up the penetration curve. The Share & Penetration view ranks all brands side by side on monthly share, penetration %, and average user visits. Nation Topline and demographic breakdowns show where a brand is over- or under-indexing relative to the category.

Share & PenetrationNation ToplineBrand Penetration by Geo · Age · Gender
03
Who is your competition?

Maps the competitive landscape through two lenses: customer profile overlap (which demographic cohorts do rival brands share?) and session flow (what share of the category's total sessions is each brand winning or losing?). The Duplication Matrix shows what % of each brand's users also use every other brand. Share of Requirements quantifies the session leakage — what is going to competitors and what is % Not Won from the total category opportunity.

Customer Profiles · Age · Gender · Intensity · GeoApp LoyaltyDuplication MatrixShare of Requirements
04
What are your growth opportunities?

Identifies the segments where a brand is under-penetrated relative to either the full category or a specific competitor. The Category Target Audience view shows which demographic groups use the category but not the brand — the highest-value acquisition pool. Brand Target Audience narrows to segments where the brand itself is weakest. Penetration Targets overlays both to produce a prioritised list of reachable, high-opportunity cohorts.

Category Target AudienceBrand Target AudiencePenetration Targets
05
Where do your users spend their time online?

Cross-references app users against social media channel visits to reveal which platforms over-index for a brand's audience versus the category average. The share gap — brand channel share minus category channel share — shows where a brand's users are disproportionately reachable, enabling precision media channel selection. A head-to-head compare mode shows how two brands' social footprints differ, identifying where a competitor has channel advantages worth closing.

Social Channel ReachCategory vs Brand Channel ShareBrand-to-Brand Compare

How Intelligence is Delivered

Omni Growth outputs reach clients through two complementary channels — structured file reports for integration into existing data workflows, and a live interactive dashboard for self-serve exploration.

📦
Report Delivery — S3 Bucket
Weekly & monthly · structured file drops

Pre-built category and brand intelligence reports are posted directly to a client-designated S3 bucket on a fixed cadence. Weekly snapshots cover session volume trends and top-line brand movement. Monthly reports include the full suite — penetration, intensity mix, Share of Requirements, App Loyalty distribution, and Social Channel Reach — pre-formatted and ready for downstream ingestion or stakeholder distribution without any manual extraction step.

Weekly snapshots Monthly full reports Client S3 bucket No manual extraction
📊
Omni Growth Dashboard
Interactive platform · 25 views · self-serve

The full Omni Growth web platform gives authorised users direct, interactive access to all 25 dashboard views across the five strategic question groups. Users can switch categories, change session types, adjust the reference month, paginate through geographic rankings, and compare brands head-to-head — all in real time without waiting for a scheduled report. The platform updates monthly alongside the underlying data refresh and is accessible via a secured web application with role-based access.

25 dashboard views Live category & brand switching Role-based access Monthly refresh

Platform in Action

Live views from the growth intelligence platform — the dashboard environment that surfaces the data outputs documented in this product.

Category Behaviour dashboard
Category Behaviour
Monthly Trend & Intensity Mix

A dual-axis chart tracking Active Users and Sessions as columns against Penetration % as a line over the last 6 months — each month's tooltip shows the exact figures. Paired below are two 100% stacked intensity bars showing the Light / Medium / Heavy split for both users and sessions. A rising Light share signals that the category is broadening its reach into occasional, new adopters, which per Byron Sharp is the primary driver of healthy, sustainable category growth.

Share and Penetration dashboard
Brand Growth
Share & Penetration

Three side-by-side horizontal bar charts rank every brand in the category on App Visit Monthly Share %, Penetration %, and Average User Visits — enabling immediate competitive positioning without switching views. The headline KPI strip (Total Sessions, Category Users, Panel Size as rolling 30-day) sits above the charts to provide the denominator context before reading the brand bars. This is the primary starting point for brand share tracking and competitive benchmarking.

Share of Requirements dashboard
Competitive Intelligence
Share of Requirements Matrix

An N×M sessions matrix quantifying what share of each brand's total category sessions are spent in their own app versus flowing to competitors. The % Not Won column captures combined leakage to all other brands in a single figure. Below, the % Sessions Lost 6-month trend reveals whether competitive pressure is building or easing over time, and the full pairwise detail table exposes every brand-combination breakdown for deeper forensic analysis.

Data Requirement

Input Data Specifications

Omni Enterprise is built on three operator data streams — web traffic signals, physical foot traffic observations, and the cell site reference master. Together they provide the engagement, location, and geographic context needed to measure how subscribers use app categories across the network.

Source 01
Web Traffic Events
Primary · Behavioural

Network-observed app and web domain visit events for every subscriber in the panel. Each row is a deduplicated session-boundary event carrying SNI/DNS host name, transfer volumes, and session duration. Primary source for category classification, session-type assignment, and monthly active user counts. Rolling lookback: 6 months. Volume: ~1.25B rows/day · ~58 GB/day compressed · ingested daily, aggregated monthly.

FieldTypeDescription
TOKEN_ID
VARCHAR(64)
SHA-256 pseudonymous subscriber identifier, re-salted nightly at midnight UTC. Primary join key across Web Traffic and Foot Traffic events. Never exposes raw subscriber identity.
START_VISIT_DATE_TIME
TIMESTAMP TZ
Session start timestamp in UTC, minute precision. Partition basis for all time-range queries — always filter on this column to avoid full-table scans.
CELL_ID
VARCHAR(20)
Network cell identifier in the format {site_code}-{year_commissioned}. Foreign key to cell_id in the Cell Site Master for geographic resolution.
VALID_FROM_CELL
TIMESTAMP
Point-in-time cell configuration timestamp. Used for temporal joins to the Cell Site Master when a cell has since been reconfigured or decommissioned within the rolling window.
HOST_NAME
VARCHAR(128)
Domain or app hostname extracted from SNI or reverse DNS. 500K+ unique values per day. App-to-category classification is applied at ingestion using this field.
DOWNLOAD_VOLUME
BIGINT
Downlink data transferred in bytes for this session. Used for data-intensity metrics and bandwidth demand analysis by category.
UPLOAD_VOLUME
BIGINT
Uplink data transferred in bytes. Elevated upload relative to download signals UGC or live-streaming behaviour within a category.
SESSION_DURATION
INT
Total TCP session duration in seconds. Used to classify sessions into the 6 defined session-intensity types (sub-15min, 15–60min, continuous 1hr+, etc.).
Hit
INT
Count of active SNI request events within the session boundary. Distinguishes background keep-alive traffic from active in-app engagement.
Aggregation pipeline: timestamped raw events (58 GB/day, Coldline) → 5-min rollup (31× row reduction) → 15-min rollup for dashboard delivery. TOKEN_ID links every web session to its corresponding foot traffic record for cross-domain subscriber journey analysis.
Source 02
Location Events (Foot Traffic)
Primary · Spatial

Network-positioning pings indicating subscriber presence at physical locations, derived from cell site attachment events with dwell-time estimation. Cross-referenced with the Cell Site Master to resolve each ping to a named geographic area. Underpins all geographic dimension tables and penetration-by-region metrics. Rolling lookback: 6 months. Volume: ~2.5B events/day after deduplication (from ~8B raw pings) · ~37 GB/day compressed.

FieldTypeDescription
TOKEN_ID
VARCHAR(64)
Same SHA-256 pseudonymous identifier as Source 01 — the primary cross-domain join key. Links a subscriber's foot traffic dwell events to their concurrent web traffic sessions.
start_time
TIMESTAMP TZ
UTC lower bound of the dwell window. Partition basis for time-range queries. Post-aggregation granularity: 5-min per device × H3-r8 hexagon (461 m cell).
end_time
TIMESTAMP TZ
UTC upper bound of the dwell window. end_time − start_time = dwell duration. Stationary pings have equal start and end coordinates.
start_latitude
DOUBLE
WGS-84 latitude of dwell start position, 15 significant figures of precision. Pairs with start_longitude as the entry coordinate for the dwell window.
start_longitude
DOUBLE
WGS-84 longitude of dwell start position.
end_latitude
DOUBLE
WGS-84 latitude of dwell end position. Equal to start_latitude for stationary pings; differs for subscribers in motion (walking, slow traffic).
end_longitude
DOUBLE
WGS-84 longitude of dwell end position.
cell_site_id
VARCHAR(20)
Network cell identifier, same format as CELL_ID in Source 01: {site_code}-{year_commissioned}. Foreign key to the Cell Site Master for geographic resolution and area code lookup.
Aggregation: 5-min per device × H3-r8 hexagon (461 m cell) → 15-min per device × H3-r7 hexagon (5.16 km² tile). The cell_site_id → postcode_area → regional district join chain through the Cell Site Master is the geographic backbone of every by-region output dataset.
Source 04
Cell Site Master
Reference · Static

Authoritative mapping of network cell identifiers to named geographic areas. The join key that converts raw cell_id and cell_site_id values from Source 01 and Source 02 into the 126 regional area codes used across all geographic dimension outputs. Updated monthly to reflect network topology changes, new site activations, and boundary revisions. Coverage: ~500K sites · ~12 MB/month compressed.

FieldTypeDescription
cell_id
VARCHAR(20)
Primary join key. Format: {site_code}-{year_commissioned}. Matches CELL_ID (Source 01) and cell_site_id (Source 02). A single physical tower may have multiple rows — one per antenna sector.
site_latitude
DOUBLE
WGS-84 latitude of the physical antenna. Used for geospatial radius queries and H3 index assignment when resolving foot traffic pings to named geographic areas.
site_longitude
DOUBLE
WGS-84 longitude of the physical antenna.
postcode_area
VARCHAR(10)
Bridge key to named geographic districts. Maps each cell site to one of 126 regional area codes used in all geographic dimension datasets. The primary geographic aggregation key for output tables.
valid_from
DATE
Date from which this cell configuration is active. Used for point-in-time joins — join on VALID_FROM_CELL BETWEEN valid_from AND valid_to to retrieve historically correct cell metadata.
valid_to
DATE
Date on which this cell configuration was superseded. NULL means the configuration is currently active. Include valid_to IS NULL in production queries to select only live site records.
technology
VARCHAR(10)
Radio access technology at this cell. Values: 4G-LTE, 5G-NR, 5G-mmWave, 3G-UMTS. Used to segment coverage analysis and benchmark engagement by network generation.
sector
TINYINT
Antenna sector number (1–6). Each physical tower site is divided into directional sectors. The combination of cell_id + sector uniquely identifies a single beam direction at a tower.
Join chain for geographic resolution: CELL_ID (Source 01) or cell_site_id (Source 02) → cell_id (Cell Site Master) → postcode_area → named district. Always use the point-in-time join (VALID_FROM_CELL BETWEEN valid_from AND valid_to) to handle cells upgraded or repositioned within the rolling 6-month window.
Cost Estimates · Category Pipeline Model

Per-Category Processing Cost: Jul 2026 – Dec 2030

Cost model built around the actual category pipeline: for each of 20 categories, identify relevant web URLs, find all MSISDNs who browsed them (50% of subscribers), pull their full web and footfall history, then aggregate daily. Each category runs independently — 20 separate scans totalling ~196 TB/day. BQ Enterprise Slots is the recommended compute platform — at $220/day (500 slots × 10hr) it is the most cost-effective option, 5.6× cheaper than BQ On-Demand at $1,225/day.

5-Yr Total · BQ Slots
~$561K
Recommended · Compute + Storage + Delivery
Daily Pipeline Cost
~$220
BQ Enterprise Slots · 20 cats
TB Scanned / Day
~196 TB
Per-Category · 20 cats × 50%
vs BQ On-Demand
5.6×
BQ Slots cost advantage
5-Year Total Cost · Per Category · Current Slider Settings
BQ Slots · Recommended
~$2.6K
per category · 5yr all-in
Dataproc Serverless
~$4.3K
per category · 5yr all-in
BQ On-Demand
~$15K
per category · 5yr all-in
Drag the Number of Categories or Coverage sliders below — this box updates live. At 50% per-category coverage with 20 categories, union coverage is effectively 100% of 82M subscribers.
BQ Enterprise Slots · Cost Breakdown by Year
Compute · Storage · Delivery — stacked. Changes when you switch Compute Option below.
GCP compute CUDs save 25–40%; drag to model discount scenarios
0%1020304050607080%
★
Key Assumptions Behind the 5-Year Cost Estimate
1
~82M subscriber base · ~82M subscribers in pipeline
Union coverage = 1−(1−50%)²⁰ ≈ 100% of 82M ≈ 82M subscribers feed into Steps 3 & 4 at default 50% coverage. Adjust via the Category / Coverage sliders above.
2
20 categories · 50% average subscriber coverage per category · per-category mode (default)
Default slider values at 50% per-category coverage. Per-category mode runs each of 20 categories separately — each scanning 41M subscribers independently. Total scan = 20 × (MSISDN + web + foot) = ~196 TB/day.
3
20% YoY data volume growth · 2026 = H2 only (184 days)
Growth applied to all scan volumes, subscriber counts, and output sizes year-on-year. Full-year costs from 2027 onward with 20% applied cumulatively.
4
Intermediate storage: 5 tables per signal type · day-level aggregated rows · 5× BQ Capacitor compression · 12-month retention
Steps 3 & 4 produce 5 tables each (web × 5, foot × 5), 1 row/sub/day, 5× BQ columnar compression. At 50% coverage per category (41M subs): 4.1 GB/day web + 3.28 GB/day foot. BQ Active ($0.020/GB/mo) for ≤6 months, Long-term ($0.010/GB/mo) for months 7–12 → ~$40/month per category intermediate storage.
5
GCP list pricing (July 2026) — no committed-use discounts applied: BQ Enterprise Slots $0.044/slot-hr · Dataproc $0.04/vCPU-hr + $1.10/TB BQ Storage Read API · BQ On-Demand $6.25/TB scanned
Drag the CUD slider above to model 25–40% negotiated discount scenarios for Dataproc or BQ Slots reservations.

Category Pipeline — 5-Step Flow (runs daily per batch of categories)

Step 1
Category URL Taxonomy
~10 GB
~$0.06
Look up URL signature list for each active category from the pre-built taxonomy table — low scan, cached.
Step 2
MSISDN Identification
~1 TB
~$7
~$0/mo stored
Scan web partition (HOST_NAME clustered) to find all MSISDNs who browsed category-relevant URLs. BQ pruning reduces scan to ~25% of partition. MSISDN set is tiny (~60 MB).
Step 3
Filtered Web Traffic
~4.0 TB
~$0
~$45/mo stored
Pull the full web browsing history only for the identified MSISDNs. Volume = union coverage × 4 TB. Retention window set by slider (default 3 months).
Step 4
Filtered Foot Traffic
~12.0 TB
~$0
~$36/mo stored
Pull footfall events for the same MSISDN set. Volume = union coverage × 12 TB foot partition. Outputs day-level aggregations (1 row/sub/day, 5× BQ compress) — at 90-day retention this step costs only ~$1.5/month stored.
Step 5
Category Aggregation
~1.0 TB
~$2
~$0/mo stored
Aggregate filtered web + footfall per category into metrics tables. Outputs are small (~20 MB/cat/day compressed). These final tables are the product deliverable.

Cost Model Parameters — drag to explore scenarios

Number of Categories 20
Each category runs its pipeline independently — N categories = N separate scans; higher isolation, cost scales linearly with N
Avg Subscriber Coverage / Category 50%
% of ~82M subscribers who browsed category URLs in a typical day. Drives Steps 3 & 4 scan volume.
Processing Mode
Separate pipeline run per category — 20 cats × 50% = 196 TB/day · $220/day BQ Slots · $1,225/day OD
Compute Option
$0.044/slot-hr Enterprise Edition — 500 slots × 10hr = $220/day (per-cat mode) · most cost-effective option
Intermediate Data Retention 365 days (12 mo)
1mo3mo6mo9mo12mo
5 intermediate tables per signal type (web × 5, foot × 5) retained per the slider. Default 12 months. Storage priced at BQ Active ($0.020/GB/mo) for ≤180 days, Long-term ($0.010/GB/mo) for days 181+. Shorter = lower storage cost; longer = richer historical reprocessing.
Pipeline StepScan VolumeDaily ComputeInt. Storage / mo (retention window)Annual All-in
Step 1 — Category URL taxonomy10 GB$0.06—$22
Step 2 — MSISDN identification1.00 TB$7.00— (~60 MB set)$2,555
Step 3 — Web day-agg (1 row/sub/day · 5 tables · 5× BQ compress)4.00 TB$0$45/mo (12mo retain)$536
Step 4 — Foot day-agg (1 row/sub/day · 5 tables · 5× BQ compress)12.00 TB$0$36/mo (12mo retain)$429
Step 5 — Category aggregation output1.60 TB$0— (product output)$0
Slot reservation (BQ Slots) / vCPU overhead (Dataproc)—$17.60—$6,424
Annual All-in (compute + storage · no growth)18.6 TB/day$18/day$80/mo$7.4K
Retention window: Daily MSISDN-level snapshots from Steps 3 & 4 are retained for reprocessing and backfill — configurable via the slider above (default 3 months / 90 days). Shorter retention reduces storage cost; longer enables deeper historical reprocessing without re-scanning raw partitions.

Annual Cost Forecast 2026–2030 · All Three Compute Options

Annual total cost by compute strategy for the current slider settings. 20% data volume growth applied per year. 2026 reflects H2 only (184 days).

Detailed Annual Breakdown · Selected Compute Option

Cost Line2026 H220272028202920305-Yr Total
Pipeline compute (Steps 1–5)$4.1K$9.8K$11.8K$14.1K$17.0K$56.8K
Output & report table storage (BQ)$0.05K$0.12K$0.14K$0.17K$0.21K$0.69K
Weekly aggregation jobs (52×/yr)$0.02K$0.03K$0.04K$0.05K$0.06K$0.20K
Monthly aggregation jobs (12×/yr)$0.01K$0.01K$0.01K$0.02K$0.02K$0.07K
S3 delivery — report exports (weekly + monthly per category)$0.09K$0.11K$0.13K$0.16K$0.19K$0.68K
BigQuery Analytics Hub — listing admin$0.10K$0.20K$0.20K$0.20K$0.20K$0.90K
Total Annual Spend (Dataproc)$4.4K$10.3K$12.3K$14.7K$17.7K$59.4K

Which Compute Is Right?

All options run the same 5-step category pipeline and produce identical output datasets. The cost difference comes entirely from how each handles the 196 TB/day of per-step data reads.

Option A · Recommended
BQ Enterprise Slots
~$561K 5-yr
  • • $0.044/slot-hr (Enterprise Edition)
  • • 500 slots × 10h/day = $220/day — lowest of all options
  • • Shared slot pool — no per-query cost once reserved
  • • ~$40/month intermediate storage per category (5 tables × 12-mo)
  • • Pure SQL, no cluster management, native BQ ecosystem
Recommended: $220/day beats Dataproc ($236/day) by $16/day — compounding to ~$40K savings over 5 years. Pure SQL, no Spark overhead.
Option B · Architectural Alternative
Dataproc Serverless (Spark)
~$601K 5-yr
  • • $0.04/vCPU-hr + $1.1/TB BQ Storage Read API
  • • 256 workers × 2h × $0.04 + ~196 TB × $1.1 = ~$236/day
  • • Each category runs as a separate Spark job
  • • No persistent cluster — spins up and tears down per run
  • • Good fit for Spark-native teams needing job-level isolation
Strong alternative: Choose Dataproc if the team is Spark-native, no BQ slot reservation exists, or strict per-category job isolation is required. Only $16/day more than BQ Slots — a near-equivalent choice on cost.
Option C · Not Recommended
BigQuery On-Demand
~$3.1M 5-yr
  • • $6.25/TB scanned — cost scales with every byte read
  • • ~$1,225/day — 5.6× more expensive than BQ Slots
  • • Easiest to implement — pure SQL, no reservation needed
  • • Scales very poorly: N categories × 196 TB × $6.25
  • • Use for ad-hoc exploration only, not daily pipelines
Not recommended: 196 TB/day × $6.25 = $1,225/day. Over 5 years: ~$3.1M vs ~$561K for BQ Slots — a ~$2.5M difference. Ad-hoc only.
Compute Option2026 H220272028202920305-Yr Total
BQ Enterprise Slots Recommended$3.3K$7.8K$9.4K$11.3K$13.5K$45K
Dataproc Serverless Arch. Alternative$4.4K$10.3K$12.3K$14.7K$17.7K$61K
BQ On-Demand High Cost$14.1K$33.9K$40.7K$48.8K$58.6K$196K
DimensionBQ On-DemandDataproc ServerlessBQ Enterprise Slots
Cost basis$6.25 / TB scanned$0.04/vCPU-hr + $1.1/TB read API$0.044/slot-hr (reservation)
Daily cost · 20 cats · per-cat mode~$1,225~$236~$220
Intermediate storage · per category~$40/month~$40/month~$40/month
5-year all-in total~$3.1M~$601K~$561K
Setup complexityLow — pure SQLMedium — Spark + orchestrationLow — SQL + slot quota mgmt
Scales with N categoriesPoorly (N × 196 TB × $6.25)Linearly (N separate Spark jobs)Well (shared slots, SQL)
VerdictAd-hoc onlyAlternative (Spark teams)✓ Recommended
BQ Slots Daily Cost
$220
500 slots × 10 hr × $0.044
Dataproc Daily Cost
$236
256 workers × 2hr + 196 TB read API
5-Yr Saving vs Dataproc
~$40K
BQ Slots ~$561K vs Dataproc ~$601K

BQ Slots wins on compute — narrowly: 500 Enterprise Edition slots × 10 hours = $220/day. Dataproc costs $236/day (256 workers × 2hr × $0.04 plus 196 TB × $1.10 BQ Storage Read API). The $16/day gap, compounded at 20% annual growth, accumulates to ~$40K in savings over 5 years. Storage is equal across all options at ~$40/month per category. The two platforms are now near-equivalent on cost.

When to choose Dataproc instead: At only $16/day difference, the decision is less about cost and more about team fit. Prefer Dataproc if (a) no BQ slot reservation exists; (b) the team is Spark-native and values Spark orchestration; or (c) strict per-category job isolation via separate Spark contexts is operationally required.

BQ On-Demand is not for production: 196 TB/day × $6.25 = $1,225/day — 5.6× more than BQ Slots. Over 5 years: ~$3.1M vs ~$561K for BQ Slots — a ~$2.5M difference. Reserve for ad-hoc exploration only.

Data Outputs

Growth Platform Datasets

Eight production datasets covering category growth diagnostics, competitive brand intelligence, and social channel reach. All datasets are derived from live network observability data, refresh monthly, and are queried directly via the shared BigQuery environment — no ETL or data movement required.

Datasets
8
5 growth · 3 competitive
Refresh
Monthly
M+5 business day availability
Session Types
6
open_app · login · commerce · more
Delivery
BigQuery
Shared dataset · read-only views
📊
Section A
Growth Diagnostics
Category panel universe and five behavioural dimension tables — the foundational layer for tracking how categories and brands are growing across the subscriber population, split by age, gender, geography, and engagement intensity.
5
Datasets
Dataset 01
Category Panel Universe
Foundation · Reference

Daily reference table tracking the total number of mobile subscribers observed by the operator's network panel — overall and split by age band, gender, and geographic area. This is the denominator for every penetration metric on the platform: when a category shows 33% national penetration, it is measured against the total panel universe from this table. The panel extends further into the future than the behavioural tables, so the newest months may show panel-only rows without a corresponding category line.

FieldTypeDescription
visited_date
DATE
Daily observation date. To derive monthly panel size, aggregate by month and take the month-end value of rolling_30d_unique_customers — not a SUM. Use DATE_TRUNC(visited_date, MONTH) to group by month.
attribute
STRING
Segmentation dimension. Values: overall (total network panel), age_band (panel by age group), gender (panel by gender), postcode_area (panel by geographic district). Filter to overall for the headline panel denominator.
attribute_value
STRING
Value within the attribute dimension. For overall: always 'all'. For age_band: 18-24, 25-34, 35-44, 45-54, 55-64, Over 65, Unknown. For gender: M, F, U. For postcode_area: 126 regional area codes mapped to named geographic districts.
daily_unique_customers
INTEGER
Number of distinct subscribers observed on this specific date within this segment. Fluctuates day-to-day due to observation gaps. Use AVG(daily_unique_customers) across a month for a stable estimate of the typical daily active panel.
rolling_30d_unique_customers
INTEGER
Count of distinct subscribers observed at least once in the 30 days ending on visited_date. This is the canonical panel size figure. Take the month-end value (MAX(visited_date) within the month) as the denominator for penetration calculations. Note: rows are absent for October 2025 — penetration calculations must handle this gap explicitly.
To compute monthly panel size: ARRAY_AGG(rolling_30d_unique_customers ORDER BY visited_date DESC LIMIT 1)[OFFSET(0)] grouped by DATE_TRUNC(visited_date,MONTH) where attribute='overall' AND attribute_value='all'. Join to category behavioural tables on month_id to compute penetration rates.
Dataset 02
Category Trends · Age Band
Growth · Demographic

Monthly category engagement metrics split by subscriber age band. Core table for tracking how a category's user base is growing across demographic cohorts over time. Each row represents one category × application × session type × age band × month combination. Filter to application = 'Any App' to get deduplicated category totals — a subscriber who uses both Uber and Bolt counts once, not twice. Summing individual-app rows will double-count multi-app users.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month (e.g. 2026-07-01). All trend analysis joins on this field. Use DATE_TRUNC(month_id, MONTH) = DATE(@month) when filtering to a specific month.
category
STRING
App category name (e.g. Ride Hailing, Food Delivery, Music Streaming). Consistent across all behavioural dimension tables.
application
STRING
App name, or 'Any App' for the deduplicated category total. Always filter to application = 'Any App' for category-level metrics — this is the pre-aggregated, deduplication-safe row. Individual app rows are valid for per-app analysis but must not be summed to produce category totals.
session_type
STRING
Engagement signal type. Values: open_app (app launched), login (authenticated session), commerce_intent (payment/checkout activity), transactional_activity (confirmed transactions), all_sessions (union of all types), driver_activity (supply-side activity where applicable). Use open_app as the default for general reach analysis.
age
STRING
Age band of the subscriber. Values: 18-24, 25-34, 35-44, 45-54, 55-64, Over 65, Unknown. The canonical display order is this sequence. Unknown includes subscribers whose age could not be resolved from billing or enrichment data.
active_users
INTEGER
Distinct subscribers in this segment who had at least one qualifying session within the month. With application = 'Any App', this is the deduplicated count of category users in this age band. Divide by the corresponding rolling_30d_unique_customers from Dataset 01 (same month, same age_band attribute) to get age-band penetration rate.
total_15min_sessions
INTEGER
Total number of sessions lasting at least 15 minutes for subscribers in this segment this month. Use for broad engagement volume analysis — includes lighter browsing and short transactional interactions.
total_1hr_cont_sessions
INTEGER
Total number of continuous sessions lasting at least 1 hour. Use for deep engagement analysis — sustained sessions indicate committed usage patterns. Particularly useful for identifying Heavy-tier users and tracking engagement depth across age cohorts.
Deduplication rule: SUM(active_users) WHERE application = 'Any App' GROUP BY month_id, age gives the correct age-band breakdown of category users. Summing individual app rows inflates the count wherever subscribers use multiple apps in the same category in the same month.
Dataset 03
Category Trends · Gender
Growth · Demographic

Monthly category engagement metrics split by subscriber gender. Structurally identical to the Age Band table with a gender column in place of age. Use to track whether a category is skewing toward or away from a specific gender over time, or to compare per-app gender profiles within a competitive set. Unknown (U) subscribers are included — this is a population statistics table, so unattributable slices are part of the picture.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month. See Dataset 02 for join conventions.
category
STRING
App category name. Consistent with all other behavioural dimension tables.
application
STRING
App name, or 'Any App' for deduplicated category total. Same deduplication convention as Dataset 02 — always filter to 'Any App' for category-level gender breakdowns.
session_type
STRING
Engagement signal type. Same six values as Dataset 02.
gender
STRING
Subscriber gender. Values: M (Male), F (Female), U (Unknown/unattributed). The underlying data includes all three values. Summing M + F + U across application = 'Any App' equals the overall active_users for that month and session type.
active_users
INTEGER
Distinct subscribers of this gender in this category this month. With application = 'Any App', deduplicated across all apps in the category.
total_15min_sessions
INTEGER
Sessions ≥ 15 minutes for subscribers of this gender in this segment this month.
total_1hr_cont_sessions
INTEGER
Continuous sessions ≥ 1 hour. Use alongside active_users to compute sessions-per-user — reveals engagement intensity differences between gender cohorts.
To compute gender-split penetration: join active_users WHERE gender IN ('M','F','U') AND application='Any App' to Dataset 01 WHERE attribute='gender' matching on attribute_value = gender and month_id. The gender codes match directly between tables.
Dataset 04
Category Trends · Geography
Growth · Spatial

Monthly category engagement metrics split by subscriber geographic area — one of 126 regional districts per row. Use to identify regional growth pockets, track whether a category's footprint is expanding geographically, or rank regions for location-targeted campaign prioritisation. Geography trend charts typically use per-app rows (excluding 'Any App') rather than the deduplicated total, as individual geography rows do not double-count users — a subscriber in one district is counted in that district regardless of how many apps they use.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month.
category
STRING
App category name. Consistent across all dimension tables.
application
STRING
App name, or 'Any App' for deduplicated category total. Geography trend charts typically exclude 'Any App' and sum per-app rows — note that individual app rows do not double-count geography.
session_type
STRING
Engagement signal type. Same six values as Dataset 02.
geography
STRING
Regional area code corresponding to a named geographic district. 126 distinct values covering the operator's full geographic footprint. Map to place names for display using a regional lookup — raw area codes are not shown to end users.
active_users
INTEGER
Distinct subscribers in this region using this category/app this month. Use to rank regions by category penetration and identify geographic growth leaders.
total_15min_sessions
INTEGER
Total 15-min+ sessions from subscribers in this region this month. High sessions relative to active users indicates a highly engaged regional user base.
total_1hr_cont_sessions
INTEGER
Total 1-hr continuous sessions from subscribers in this region. Useful for identifying regions with sustained, committed engagement as distinct from high-frequency but short sessions.
For top-region trend analysis: rank regions by SUM(active_users) WHERE application != 'Any App' over a 6-month window, then retrieve monthly series for the top-N regions. Apply client-side pagination (15 regions per page) to avoid chart overload. The trend window is set client-side by slicing the full history to the last 6 months from the selected end month.
Dataset 05
Category Trends · Engagement Intensity
Growth · Behavioural Tier

Monthly category engagement metrics split by subscriber intensity tier — Light, Medium, or Heavy users. Per Byron Sharp's market growth principles, healthy category growth is primarily driven by Light users (the broad, occasional audience). This table enables clients to monitor whether growth is coming from Light-tier acquisition or Heavy-tier deepening. Classification is pre-applied in the source data; no threshold parameters need to be defined.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month.
category
STRING
App category name.
application
STRING
App name, or 'Any App' for deduplicated category total. For intensity mix charts, always filter to application = 'Any App' to avoid double-counting multi-app users across tiers.
session_type
STRING
Engagement signal type. Note: transactional_activity only has Light tier data for Ride Hailing; login has no intensity rows at all — this is an upstream data characteristic, not a pipeline error.
intensity
STRING
Engagement intensity tier: Light (occasional users), Medium (regular users), Heavy (power users). Classification thresholds are pre-applied at source. The canonical display order is Light → Medium → Heavy.
active_users
INTEGER
Distinct subscribers in this intensity tier for this category/month. With application = 'Any App', the three tier rows sum to the category's total active users for that session type and month. Compute tier share as tier_users / SUM(tier_users OVER all tiers) for a given month and session type.
total_15min_sessions
INTEGER
Sessions ≥ 15 minutes in this intensity tier. Heavy users typically generate a disproportionately large share of total sessions despite being a small fraction of the user base.
total_1hr_cont_sessions
INTEGER
Continuous sessions ≥ 1 hour in this intensity tier. A high Heavy-to-Light ratio here may indicate the category is niche and deeply engaged rather than broadly penetrating.
For intensity mix visualisation: filter to application = 'Any App', group by month_id, intensity, and compute share as each tier's active_users divided by the month total. Render as a 100% stacked bar in Light → Medium → Heavy order. A growing Light share over time is the primary healthy-growth signal per market growth theory.
🏆
Section B
Competitive & Channel Intelligence
App loyalty profiles, cross-brand user overlap matrices, and social channel reach — the competitive intelligence layer for market share analysis, brand duplication mapping, and channel targeting.
3
Datasets
Dataset 06
App Loyalty Profile
Competitive · Brand Loyalty

Monthly snapshot of subscriber loyalty tiers within a category — Solus (used only one app), Dual (used exactly two apps), or Three+ (used three or more apps). Pre-classified in the source data. Shows how exclusively or promiscuously subscribers engage within a category and which apps attract the most loyal versus multi-homing users. An app with a high Solus share has an exclusive user base; a high Three+ share indicates the app is commonly used alongside competitors and may be a secondary choice.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month. Loyalty is a monthly snapshot — a subscriber's tier is classified based on all apps used within that calendar month.
category
STRING
App category. Loyalty classification is category-scoped — a subscriber who uses one Ride Hailing app and two Food Delivery apps is Solus in Ride Hailing and Dual in Food Delivery simultaneously.
application
STRING
App name. Unlike the growth tables, the per-app breakdown (which apps attract Solus vs. Dual vs. Three+ users) is the primary analytical output. Exclude 'Any App' rows for per-app loyalty stacked bar analysis. Use application != 'Any App' when computing category-level loyalty distribution.
session_type
STRING
Engagement signal type. Default for loyalty analysis is all_sessions — this captures the full picture of multi-app usage regardless of session depth. Using open_app alone may miss transactional users who only appear in commerce signals.
app_loyalty
STRING
Loyalty tier: Solus (subscriber used only this one app in the category that month), Dual (used exactly two apps), Three+ (used three or more apps). Pre-classified at source — no threshold parameters to define.
active_users
INTEGER
Distinct subscribers in this app × loyalty tier × month combination. For category-level loyalty distribution, sum across all applications excluding 'Any App' and group by app_loyalty.
total_15min_sessions
INTEGER
Sessions ≥ 15 minutes for this loyalty cohort. Three+ users typically generate more total sessions despite being a smaller group — use a dual-axis view to make this visible alongside user counts.
total_1hr_cont_sessions
INTEGER
Continuous sessions ≥ 1 hour for this loyalty cohort. Heavy session depth in the Three+ tier suggests the app is used as a primary option even among multi-homing subscribers.
For per-app loyalty stacked bar: filter to application != 'Any App', group by application, app_loyalty, render as 100% stacked in Solus → Dual → Three+ order. For category-level distribution: sum active_users across all apps (excluding 'Any App') grouped by app_loyalty. The default session_type for loyalty analysis is all_sessions.
Dataset 07
Brand Overlap & Share of Requirements
Competitive · Market Structure

Monthly cross-app user overlap matrix — the structural competitive intelligence dataset. Each row represents a pair of apps within a category and quantifies what fraction of App X's users also used App Y in the same month. Enables N×N duplication heatmap visualisation, share-of-requirements analysis, and identification of which competitors are capturing the most shared users. The diagonal (same-app pairs) is excluded from the raw data and is conventionally rendered as 100% client-side.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month. Overlap is computed as a monthly snapshot — all user counts refer to distinct subscribers active in the calendar month.
category
STRING
App category. Both application_x and application_y belong to this category. Filter to a single category for within-category competitive analysis.
session_type
STRING
Engagement signal type applied to both apps in the pair. Use open_app for broad competitive overlap, or commerce_intent / transactional_activity for transaction-level competitor intelligence.
application_x
STRING
Source app — the app whose user base is being measured. Read each row as: "X% of application_x's users also used application_y this month." Each app appears as application_x once for every other app in the category.
active_users_x
INTEGER
Total distinct users of application_x this month. The denominator for overlap percentage: overlap_pct = ROUND(SAFE_DIVIDE(active_users_xy, active_users_x) * 100, 2).
application_y
STRING
Destination app — the competitor app into which overlap is measured. The matrix is not symmetric: overlap from App A into App B may differ from App B into App A, because both apps have different total user base sizes.
active_users_xy
INTEGER
Distinct users who used both application_x and application_y in the same month. SAFE_DIVIDE(active_users_xy, active_users_x) * 100 gives the percentage of App X's users who also use App Y.
active_users_category
INTEGER
Total deduplicated users of the entire category this month — equivalent to active_users WHERE application = 'Any App' in the growth tables. Use as the denominator for share-of-requirements: what fraction of the category audience is captured by the shared users of any given app pair.
total_15min_sessions_x
INTEGER
Total 15-min+ sessions for application_x users. Available alongside the equivalent _xy and _category variants for session-volume share-of-requirements calculations.
total_1hr_cont_sessions_x
INTEGER
Total 1-hr continuous sessions for application_x users. Use for deep-engagement share-of-requirements analysis.
For the N×N heatmap: extract distinct apps from both application_x and application_y, compute overlap_pct = ROUND(SAFE_DIVIDE(active_users_xy, active_users_x)*100,2) for each off-diagonal cell, then add 100% diagonal cells client-side. For share-of-requirements: SAFE_DIVIDE(active_users_xy, active_users_category)*100 shows the share of the total category audience captured by both apps simultaneously.
Dataset 08
Social Channel Reach
Channel · Media Planning

Monthly cross-platform overlap between mobile app users and social media channels — measuring which social platforms reach a brand's or category's user base. Answers: "Which social channels over-index for my brand's users versus the category average?" and "How does Brand A's social footprint compare to Brand B's?" There is no session_type filter on this table — social channel visits are session-type-independent and the column is absent from this source.

FieldTypeDescription
month_id
DATE
Partition key. First day of the month. Social channel overlap is measured as a monthly snapshot across all days in the calendar month.
application_x
STRING
The reference app or universe. Use 'ALL Universe' for the full category-universe view (all users of any app in the category); use a specific app name (e.g. 'Uber') for per-brand channel analysis. The category's brand list is derived by cross-referencing distinct application_x values here against the active-user rows in the growth tables for the selected category.
channel_y
STRING
Social media channel name (e.g. YouTube, Instagram, TikTok, Facebook, Snapchat, X). Full channel list is the set of distinct values in this column for the selected month.
active_users_xy
INTEGER
Distinct subscribers who used both application_x and channel_y in the same month. Raw count — must be normalised to a share percentage before comparing across brands or against the category universe. Normalise as: share_pct = active_users_xy(channel) / SUM(active_users_xy OVER all channels) × 100.
total_sessions_xy
INTEGER
Total sessions recorded in channel_y by application_x users. Complementary to active_users_xy — high sessions relative to users indicates the shared audience is highly active on that social channel, not just incidentally reached.
No session_type column — query directly by month_id and application_x. For category-vs-brand comparison: fetch once with application_x = 'ALL Universe' and once per brand, normalise both to share percentages, then compute gap = brand_share_pct − category_share_pct per channel. A positive gap means the brand over-indexes on that channel versus the category average.

Platform Summary

📄 View Full Summary

Plain-English guide to all cost models, assumptions, strategies, and caveats across every product page.