Checking access…

GCP → AWS · Secure Transit
End-to-End · Data Distribution

Distribution Architecture

US Telco data reaches Eureka via two paths: SFTP over SSH for batch file transfers, and Google Analytics Hub for direct BigQuery-to-BigQuery data sharing. Data landing in Eureka's GCP BigQuery replicates to AWS S3. Clients pull directly from S3 via STS AssumeRole temporary credentials.

Architecture

End-to-End Data Flow

① TELCO · GCP ② SECURE TRANSIT ③ EUREKA · AWS S3 ④ CLIENTS ⑤ EUREKA · GOOGLE CLOUD ANALYTICS EUREKA-MANAGED PRODUCT UIs Google Cloud Platform Web Traffic ~750M rows/day · Parquet Foot Traffic ~250M pings/day · Parquet Demographics 60M subscribers · monthly PIPELINE GCS Bucket Daily Parquet export staging Dataflow / Pub·Sub Scheduled · 02:00 UTC trigger 🔐 Cloud KMS · CMEK · AES-256 AGGREGATED DATA ONLY PRIMARY SFTP over SSH · PGP SSH Key Auth PGP-signed payloads SHA-256 Integrity S3 eureka-data-lake.s3.amazonaws.com VPC Only MFA Delete No Public Obj. Lock Omni-Alpha Quant & Web Signals /omni-alpha/YYYY-MM-DD/ 📁 US Telco Data UK · US · TH · KSA 📁 Third Party Mastercard 📁 Alpha Files Product outputs IAM: omni-alpha-{client}-reader Omni-Growth Enterprise & Retail Insights /omni-growth/YYYY-MM-DD/ 📁 US Telco Data UK · US · TH · KSA 📁 Third Party Mastercard 📁 Alpha Files Product outputs IAM: omni-growth-{client}-reader Omni-Pinpoint Location & Mobility Analytics /omni-pinpoint/YYYY-MM-DD/ 📁 US Telco Data 📁 Third Party 📁 Alpha Files IAM: omni-pinpoint-{client}-reader Daily pull · STS AssumeRole Client 1 Omni-Alpha · STS AssumeRole Quant signals · web + foot traffic STS AssumeRole 📁 Reads: /omni-alpha/{date}/ Client 2 Omni-Growth · STS AssumeRole Market share · foot traffic STS AssumeRole 📁 Reads: /omni-growth/{date}/ Client 3 Omni-Growth · STS AssumeRole Risk profiling · mobility trends STS AssumeRole 📁 Reads: /omni-growth/{date}/ Client 4 Omni-Pinpoint · STS AssumeRole Trade area · visitation benchmarking STS AssumeRole 📁 Reads: /omni-pinpoint/{date}/ Omni-Alpha UI Client Analytics Signal browsing Column masking Export controls BQ: eureka.omni_alpha Analytics Hub Omni-Growth UI Enterprise Analytics Foot traffic Web indexing Row-level security BQ: eureka.omni_growth Analytics Hub Omni-Pinpoint UI Location Intelligence Geospatial queries Trade area analysis k-anon suppression BQ: eureka.omni_pinpoint Analytics Hub Analytics Hub · BQ Data Sharing GCP → S3 sync BigQuery Data Lake — Eureka Analytics Replicated from Eureka S3 Column-level security IAM per product Analytics Hub Row-level security VPC Service Controls Datasets: eureka.omni_alpha · eureka.omni_growth · eureka.omni_pinpoint GCP
Telco GCP
Secure Transit
Eureka S3 + GCP Analytics
Clients
S3 ↔ BQ replication (dashed)
Analytics Hub · BQ Data Sharing
⚠ Aggregated data only — no raw PII crosses the transit boundary
Zone 1 · Telco GCP

Data Sources & Pipeline

Three telco datasets aggregate daily in Telco's GCP environment via Dataflow, land in GCS as Parquet, and leave at 02:00 UTC via the secure channel. Cloud KMS CMEK encrypts everything at rest before it moves.

Raw Data

Web Traffic

~750M rows/day. First-party browsing signals, Parquet-encoded, partitioned by date.

Raw Data

Foot Traffic

~250M pings/day. GPS + network location signals for venue-level visitation.

Raw Data

Demographics

60M subscribers, monthly refresh. Age, income, zip, segment — enrichment layer for all products.

Pipeline

Dataflow / GCS

5-min micro-batch aggregation. Daily export trigger at 02:00 UTC into staging GCS bucket. CMEK AES-256 throughout.

Zone 2 · Secure Transit

Secure Channel

Data never touches the public internet. SFTP over SSH with PGP-encrypted payloads is the primary transfer method. SHA-256 checksums verify every file on receipt.

1

SFTP over SSH

SSH Key Auth — PGP-signed payloads. No public internet routing.

›
2

PGP Verify

Every payload signed. Signature verified on receipt before processing.

›
3

SHA-256 Verify

Checksum on every file on receipt. Mismatch triggers alert + retry.

›
4

S3 Write

KMS-SSE applied at write. MFA Delete active. VPC-only endpoint.

Aggregated data only. Raw PII never crosses the transit boundary. US Telco's Dataflow pipeline produces pre-aggregated Parquet files (device-level signals already k-anonymized and date-partitioned) before the GCS export. Individual subscriber records remain inside US Telco's GCP perimeter at all times.
Zone 3 · Eureka AWS S3

S3 Data Lake — IAM-Isolated Prefixes

Three prefixes — one per product. But each client that accesses a product gets their own dedicated IAM role scoped to that prefix. Ten clients on Omni-Alpha means ten roles, all pointing at /omni-alpha/ — individually revocable, individually auditable. Bucket-level controls apply to all: VPC-only endpoint, KMS-SSE, MFA Delete, Object Lock (COMPLIANCE mode), no public access.

ProductS3 PrefixIAM Role PatternRoles CreatedActive WindowArchive Policy
Omni-Alpha/omni-alpha/YYYY-MM-DD/omni-alpha-{client}-reader1 per client12 monthsGlacier IR → Deep Archive
Omni-Growth/omni-growth/YYYY-MM-DD/omni-growth-{client}-reader1 per client12 monthsGlacier IR → Deep Archive
Omni-Pinpoint/omni-pinpoint/YYYY-MM-DD/omni-pinpoint-{client}-reader1 per client12 monthsGlacier IR → Deep Archive

Why one role per client — not one role per product?

✗ One shared role per product

  • Can't revoke one client without breaking all others
  • CloudTrail shows the role, not which client called it
  • Credential leak from one client compromises all

✓ One role per client per product

  • Revoke a single client instantly — others unaffected
  • Every API call tagged to a specific client in CloudTrail
  • Blast radius of any compromise limited to one client
All roles share the same s3:prefix condition pointing to the same product prefix — the prefix controls what data, the role controls who can access it.
Zone 4 · Client Access

Client Access Models

No client ever receives a long-lived AWS access key. Access is granted through one of two models: STS AssumeRole for standard per-client access, and S3 Access Points for clients requiring multiple products simultaneously.

1

STS AssumeRole

AWS-native clients · Recommended

Each client gets their own IAM role in Eureka's AWS account, with a trust policy allowing only that client's AWS account to assume it. They call sts:AssumeRole to get temporary credentials valid for 1–12 hours. No static keys ever leave Eureka.

  • One role per client per product — never shared
  • Trust policy locks to that client's AWS account ID only
  • Scoped to their prefix via s3:prefix condition
  • Revoke one client instantly — others unaffected
  • Full CloudTrail audit tagged per client role
2

S3 Access Points

Multi-product clients · Network isolation

When a client needs access to multiple products (e.g., Growth + Pinpoint), each gets a dedicated S3 Access Point with its own ARN, bucket policy, and network control — fully isolated from other clients and other products.

  • One Access Point per client × product pair
  • IP allowlist or VPC-only restriction per AP
  • Policy changes per-client — main bucket policy untouched
  • Composable with Tier 1 (AssumeRole)
  • Scales to 10,000 APs per bucket

STS AssumeRole Flow

Client AWS Account
e.g. client AWS account
→
sts:AssumeRole
cross-account API call
→
omni-alpha-
{clientname}-reader
1 role per client · in Eureka's account
→
Temp Credentials
AccessKeyId · SecretKey · Token
TTL: 1–12 hr, then expire
→
s3:GetObject
/omni-alpha/ only
VPC-only · prefix-scoped

Client × Access Method Matrix

ClientProduct(s)Access MethodHas AWS Account?Credential TTLMulti-Product?
Client 1 Omni-Alpha Tier 1 — STS AssumeRole Yes4 hrNo
Client 2 Omni-Growth Tier 1 — STS AssumeRole Yes4 hrNo
Client 3 Omni-Growth Omni-Pinpoint Tier 2 — S3 Access Points Yes4 hrYes — 2 APs
Client 4 Omni-Pinpoint Tier 1 — STS AssumeRole Yes4 hrNo
Zone 5 · Eureka Google Cloud

Analytics Environment — BigQuery

Data arrives in Eureka's BigQuery via two paths: Analytics Hub direct data sharing from US Telco's GCP, and replication from Eureka's AWS S3. All data stays inside Eureka's Google Cloud project — column-level security, row-level access policies, VPC Service Controls, and Data Catalog lineage applied across every dataset. BigQuery feeds all three product UIs via Analytics Hub grants — no data copy leaves the platform.

Omni-Alpha UI

Client Analytics

Dataset eureka.omni_alpha. Signal browsing, custom date ranges, column masking. Access-tier scoped to subscriber.

Omni-Growth UI

Enterprise & Retail Analytics

Dataset eureka.omni_growth. Category foot traffic, web indexing, demographic cross-tabs. Row-level security per client.

Omni-Pinpoint UI

Location Intelligence

Dataset eureka.omni_pinpoint. Geospatial queries, trade area analysis. k-anonymity suppression below 50 devices/venue/day.

Security controls active in Eureka Google Cloud

VPC Service Controls perimeter Column-level security Row-level access policies Analytics Hub IAM grants Data Catalog + lineage Cloud Audit Logs CMEK encryption at rest DLP scan on ingestion
Governance · Telco Transparency

How Eureka Stays Transparent to Telco

Every data movement, access event, and lifecycle change is observable and verifiable by Telco, with contractual audit rights available at any time.

Transparency pillars

01 · Annual · Independent
Third-Party Audit Right

The DPA gives Telco the right to appoint an independent auditor annually, with access to any 90-day CloudTrail + BigQuery audit window on request.

02 · On Demand · Foundation
Shared Audit Logs

AWS CloudTrail + S3 Access Logs streamed to Telco's SIEM via Kinesis Firehose (<60 s). Every GetObject, AssumeRole, and PutObject is tagged with timestamp, requester identity, and IP.

Transparency cadence

PillarFrequencyDelivery channelContents
01 · Third-Party AuditAnnual + on demandIndependent auditor appointed by TelcoFull DPA review; any 90-day CloudTrail + BQ audit window
02 · Shared Audit LogsOn demandTelco SIEM via Kinesis FirehoseRequester, object key, HTTP status, source IP, timestamp

Controls Telco holds unilaterally

  • Disable Dataflow export trigger instantly — kills the pipeline at source
  • Revoke GCS-to-S3 transfer IAM permission without notice
  • Request immediate deletion of any prefix or all data
  • Trigger a forensic audit of any 90-day period at any time

What Eureka commits to

  • No schema changes to exported datasets without 30-day written notice
  • No new clients added to a prefix without Telco notification
  • Aggregation logic changes subject to joint review before deployment
  • Deletion confirmations provided within 7 days of any purge request

Platform Summary

📄 View Full Summary

Plain-English guide to all cost models, assumptions, and caveats across every product page.