There’s been a lot of press coverage about Google Chrome’s recent announcement that it’ll stop supporting third-party cookies by 2022. This change will bring an end to an era of digital advertising that publishers have known since the mid-2000s.

Chrome’s announcement is the death knell of third-party cookies. It’s also another sign that the future of display advertising will truly be privacy first. 

Even though Chrome’s changes won’t come into effect for a while yet, privacy laws like the GDPR and privacy settings in web browsers like ad blockers, Safari’s ITP, and Firefox’s ETP have been impacting publisher ad revenues for years.

Here’s a brief look back at events that have shaped the privacy narrative in display advertising so far and the impact they’ve had on publishers.

Ad Blockers

Publishers have been dealing with ad blockers (plugins for web browsers) since the mid-2000s and have seen the rate of adoption rise over the years. 

Most ad blocker plugins work by blocking JavaScript originating from domains on a blacklist and stopping elements, such as class=”advertisement” and alt=”ad”, from loading. The latter means that ad tags from AdTech platforms (e.g. ad servers, DSPs, and ad networks) aren’t able to load, resulting in ad revenue loss for publishers. 

Because most ad blockers stop AdTech JavaScript tags from loading, they not only stop ads from being displayed but also stop third-party cookies from being created.

The impact of ad blockers on publishers is immediate, severe, and costly — lost ad revenue from ad blockers is many billion dollars per year.

Privacy Laws (e.g. the GDPR)

It’s almost two years since the European Union’s General Data Protection Regulation came into force and the impact on publishers has been mixed. 

For those that have taken the steps to comply with the GDPR, such as by selecting the correct lawful basis for data processing for advertising (i.e. consent and not legitimate interest) and implementing proper consent collection processes, the impact is akin to ad blockers and privacy settings in web browsers — i.e. smaller addressable audiences for advertisers, resulting in less ad revenue for publishers.

For those publishers, however, who have chosen not to correctly comply with the GDPR have seen a less harmful impact. Examples of these non-compliant activities include assumed consent, denying users access to their website unless they opt in to data processing, and firing tags from their advertising partners even after a user has rejected the data-processing request.

Privacy Settings in Web Browsers

Over the past few years, Safari and Firefox have made gradual changes to how their web browsers handle cookies and other storage and tracking methods (e.g. device fingerprinting) to strengthen privacy for users. 

Safari first started stepping up their privacy game in 2015 by allowing iOS users to install content blockers. 

These content blockers can be downloaded from the App Store and used to prevent certain content (e.g. ads) and tracking cookies from loading in the Safari web browser on Apple smartphones and tablets. 

Then in 2017, Apple introduced Intelligent Tracking Prevention (ITP) to further strengthen user privacy in Safari. 

ITP not only blocks third-party cookies by default, but also limits how long first-party cookies and data stored in local storage can last. The former impacts how audience targeting and retargeting via RTB works, whereas the latter impacts workarounds that AdTech companies have created to identify users without relying on third-party cookies.

Firefox adopted a similar approach in 2019 when they started blocking third-party cookies by default and restricting other user identification methods like device fingerprinting as part of their Enhanced Tracking Prevention (ETP) feature.

With Google Chrome’s recent announcement that they’ll stop supporting third-party cookies by 2022, the outlook is bleak for all parties involved in display advertising and the industry will need to undergo some major fundamental changes. 

This has been good news for Internet users, but bad news for publishers, advertisers, and AdTech companies. 

The end of third-party cookies means publishers, AdTech companies, and advertisers will need to come up with solutions that allow them to run personalized and targeted advertising, while respecting user privacy and complying with privacy laws.

How Can Publishers Survive in a Privacy-First World?

Despite the strong headwinds, there are a few things publishers can do to survive in a privacy-first world.

1. Start building addressable audiences based on first-party data and offer them to advertisers for targeting.
2. Build stronger relationships with brands and look at adopting an ID resolution service based on email IDs.
3. Ensure your consent management platform complies with the GDPR.
4. Ask your AdTech partners (e.g. SSPs) what they are doing to adapt and what their plans are once third-party cookies disappear from the three most popular web browsers.

This article was originally published on WNIP. 

Data & Privacy

What Does the EU’s DSA and DMA Mean for AdTech?

Mike Sweeney, +1

Oct 10, 2022

Data & Privacy

The Demise of Third-Party Cookies in AdTech: Why Are They Being Phased Out?

Mike Sweeney, +1

Jan 26, 2023

Data & Privacy

What’s the Difference Between Zero-Party, First-Party and Third-Party Data?

Natalia Figas

Mar 15, 2023

The post The Impact of Privacy Laws (e.g. GDPR) and Privacy Settings on Publishers appeared first on Clearcode.

The decline of third-party cookies has been common knowledge for publishers, agencies, AdTech companies, and advertisers for many years now.

The reason Chrome’s announcement has caused such a stir is that it’s the final nail in the coffin for third-party cookies — and it’s a massive nail at that. 

As the leader in the web browser market, Chrome’s decision to stop supporting third-party cookies will have a huge impact on how display and native advertising campaigns are run and measured.

How Are Third-Party Cookies Used for Display and Native Advertising?

Although the form and function of display ads vary to native ads, they are both served in a similar way. 

When a web page loads, it sends various requests to the publisher’s web server and other servers to retrieve the main content. If the page also contains a place for a display or native ad, then a request is sent off to an advertising technology (AdTech) platform to retrieve the ad.

When the AdTech platform returns the ad, it creates a cookie and stores it on the user’s device. Because this cookie was created by the AdTech platform and not the publisher, it’s called a third-party cookie.

Once a third-party cookie has been assigned to a user’s browser, it can power a number of key programmatic advertising processes, such as:

Identification: AdTech platforms like SSPs and DSPs use third-party cookies to identify users across the web.

Behavioral targeting and retargeting: Once an AdTech platform can identify a user, they can show them personalized ads based on their behavior and interests. 

Audience activation via DMPs: Advertisers can use a DMP to create audiences, which can then be exported to DSPs via cookie syncing and used for audience targeting across different websites.

Frequency capping: Third-party cookies help AdTech companies limit the number of times a user is shown the same ad.

Attribution: AdTech platforms use third-party cookies to attribute ad views to conversions. 

The Slow Decline of Third-Party Cookies in Programmatic Advertising

The decline of third-party cookies has been in free fall for the past decade, and when Google Chrome shuts off support for them, they’ll come crashing to the ground.

Here’s a recap on the decline of third-party cookies:

Ad blockers: First created in the mid-200s, ad blockers have since grown in popularity. A recent report by Blockthrough states there are around 763 million monthly active users of ad blockers on desktop and mobile devices. 

Most ad blockers prevent tags from AdTech companies from loading on a page, meaning they can’t create third-party cookies.

The GDPR: Since the GDPR’s enforcement date of May 25, 2018, publishers have had to ask EU visitors to give consent to collect their personal data, which includes creating and reading third-party cookies on their device. 

Although the GDPR’s impact isn’t as direct as ad blockers or privacy settings in web browsers, it has reduced the number of available audiences, especially in Europe. 

Web browsers: Back in 2015, Safari started allowing iOS users to install content blockers. Then in 2017, they introduced a new privacy feature called Intelligent Tracking Prevention (ITP), which not only blocks third-party cookies by default but also limits the lifespan of certain first-party cookies and data storage. 

Firefox followed suit in 2019 when it started blocking third-party cookies by default. 

Safari and Firefox’s global web browser market share stands at 22% across all devices, meaning that 22% of all website traffic is blocking third-party cookies (not including ad blockers). 

As Google Chrome’s global market share is 64%, when it stops supporting third-party cookies, it will ultimately spell the end of third-party cookies forever. 

Display and Native Advertising: Life Without Third-Party Cookies

Third-party cookies have been the backbone of the display and native advertising industry for over a decade. 

Adjusting to life without them will be hard, but there are options available. 

Google Chrome’s Privacy Sandbox

Instead of simply turning off third-party cookies like Safari and Firefox did, Chrome has proposed a replacement — one that still maintains an ad-supported web but also protects user privacy. 

And no, it’s not a browser ID. 

The replacement is known as Privacy Sandbox, and although it’s still being developed, the concept has been pretty well thought out.

Privacy Sandbox contains various APIs that will replace many of the key programmatic advertising processes currently powered by third-party cookies. 

Ad conversion measurement: There are two APIs that AdTech companies can use to receive data about click-through conversions and ad campaign reports. The key thing about these APIs is that they won’t contain any identifiers that can be used to link a click or ad view to a user. 

Ad targeting: Privacy Sandbox proposes three main methods for ad targeting — contextual and first-party-data, interest-based targeting, and remarketing (aka retargeting). Similarly, with the measurement and reporting APIs, these three ad targeting methods won’t rely on identifiers.

As you can see, identification will no longer be done on an individual basis. Instead, ad targeting will be done in on a cohort level, with reporting done in aggregate. 

AdTech companies should look into how their platforms can utilize the Privacy Sandbox APIs for display and native advertising. 

Other Alternatives

It’s clear that Google Chrome wants to move away from identifying individuals towards a more privacy-friendly solution. But what about the other main web browsers?

Many folks in the programmatic advertising industry have suggested that Safari and Firefox may adopt Chrome’s Privacy Sandbox at some stage, but this is just speculation. 

At the moment, the future of display and native advertising lies in first-party data.

Publishers, mainly premium publishers, are in the best position to weather this storm. They have direct access to users and can collect valuable first-party data about their behavior and interests. 

However, unlocking this first-party data for ad targeting and measurement isn’t as simple as with third-party cookies. 

One of the main ideas being put forward involves using more persistent IDs, such as hashed and encrypted email addresses, to tie a publisher’s first-party data to a user, and even match it with an advertiser’s first-party data. 

This would allow publishers to identify users on their site, which could power ad targeting, measurement, and attribution. 

But using such an explicit piece of personally identifiable information (PII) doesn’t follow the privacy trend in AdTech — in fact, it goes in the other direction. 

Other ideas include utilizing a browser’s local storage to store and retrieve first-party data for targeting, and exploring data clean rooms for measurement.

Businesses, especially in the CPG realm, are turning to QR Codes and NFC tags on the product to acquire first party data directly from users. Both these technologies do not require an app on most smartphones and with QR Codes – the level of consumer awareness is at an all time high. Using a QR Code Generator, businesses can easily create custom QR Codes to acquire demographic data, exact GPS location and use it to retarget users.
 
Regardless of which option your AdTech company chooses, one thing is for certain — if you haven’t started preparing your tech for a world without third-party cookies, then now is a good time to start. 

This article was originally published on MarTech Advisor

Data & Privacy

The Role of Third-Party Cookies in Programmatic Advertising and AdTech

Mike Sweeney, +1

Mar 18, 2021

Data & Privacy

6 Alternatives to Third-Party Cookies and Mobile IDs in AdTech

Mike Sweeney

Jan 23, 2023

Data & Privacy

The Demise of Third-Party Cookies in AdTech: Why Are They Being Phased Out?

Mike Sweeney, +1

Jan 26, 2023

The post How Will Display and Native Advertising Work Without Third-Party Cookies? appeared first on Clearcode.

We recently worked with one of our MarTech development teams to produce an infographic that illustrates just that.

Click on the infographic to enlarge it.

What Is a Customer Data Platform (CDP)?

A customer data platform (CDP) is a piece of marketing technology that collects and organizes data from a range of online and offline sources.

With a CDP, marketers can view detailed analytics reports, create user profiles, audiences, segments, and single customer views, as well as improve advertising and marketing campaigns by exporting the data to other systems.

Data Collection

A CDP can collect data from a range of different sources, such as websites, mobile applications and customer relationship management systems (CRMs).

The data is collected via application programming interfaces (APIs), event trackers (e.g. JavaScript tags and SDKs), server-to-server integrations and manual imports.

Data Normalization and Enrichment

Because data is collected from different sources, it often enters a CDP in different formats. The data normalization process organizes the data into a common format, removes redundancies, and transforms it into the CDP’s schema. 

The enrichment process makes the data relevant for users (e.g. marketers) by turning an IP address into a geolocation, extracting IDs from cookies, and taking the device type from the user agent string.

Analytics Reports

One of the main features of a CDP is analytics. Because CDPs collect data from a range of sources, they can produce advanced analytics reports that can provide insights into consumer behavior and engagement. 

These reports can then help marketers and advertisers improve the performance of their campaigns and increase conversions on their websites.

Profiles, Audiences, and Segments

When CDPs receive new batches of data, they create new user profiles or update existing ones via profile merging. These profiles contain data about individual users, such as which devices they use, their location, and website- or app-engagement history (e.g. pages visited and items purchased). 

From there, marketers can create segments in the analytics dashboard, and audiences, which include profiles containing common attributes like location and behavior.

Single Customer View (SCV)

CDPs create a single customer view, also known as a 360 or unified customer view, by combining all the data they have about their customers and producing a single record. 

Marketers can analyze the SCVs in their CDP to identify how users interact with their brand across different channels and to power customer service and marketing messages.

Audience Activation

Audience activation is often the end goal of a CDP; it’s where marketers can put the data they’ve collected to work and improve the performance of their advertising and marketing activities. 

There are many uses cases of audience activation, but most marketers will use the audiences they’ve created in their CDP to improve media buys via demand-side platforms (DSPs), send personalized and timely emails, run retargeting campaigns on social media platforms, improve targeting on search advertising campaigns, and perform onsite personalization (e.g. content or product recommendations).

MarTech

What is a Customer Data Platform (CDP) and How Does It Work?

Maciej Zawadziński, +1

Mar 30, 2018

Data & Privacy

Top Customer Data Platform (CDP) and Data Management Platform (DMP) Companies [Updated in 2024]

Paulina Zawiślak, +1

Jan 11, 2021

Data & Privacy

The Anatomy of a Data Lake [infographic]

Mike Sweeney

Mar 17, 2022

The post The Anatomy of a Customer Data Platform (CDP) [infographic] appeared first on Clearcode.

Build vs Rent vs Buy: A Decision Tree

Many companies that enquire about our AdTech and MarTech development services often ask us about whether they should build, rent or buy an AdTech or MarTech platform.

To help our partners, as well as other tech companies, brands, agencies, and publishers, with making this decision, we’ve created a simple decision tree:

Who Benefits from Building, Renting or Buying an AdTech or MarTech Platform?

Who benefits from building an AdTech or MarTech platform?

• AdTech and MarTech companies.
• Tech companies from other industries.
• Medium- and large-sized publishers and agencies.
• Media companies.

Who benefits from renting an AdTech or MarTech platform?

• Brands and small- and medium-sized agencies.
• Small- and medium-sized publishers.

Who benefits from buying an AdTech or MarTech platform?

• Large companies, e.g. telcos.
• Private equity companies.

What Does It Mean to Build, Rent and Buy an AdTech or MarTech Platform?

The topic of building, renting, and buying AdTech and MarTech platforms has been around for over a decade and raises its head from time to time, but there is often some confusion about what these terms mean.

Below is an overview of how we interpret these terms:

What Does Building an AdTech or MarTech Platform Mean?

This relates to software development where you build the platform, usually from the ground up. This includes designing the UX/UI (i.e. the user interface), writing the code (both backend and frontend) and testing it, and designing the infrastructure’s architecture.

The time it takes to design and build the minimum viable product (MVP) of an AdTech or MarTech platform varies depending on a number of factors, including:

• Budget and timeframes: A large budget will mean that more developers (e.g. 6+) can work on the project, resulting in the MVP’s release sooner. Essentially, the more developers working on the project the shorter it will take to build.
• Features: The more features you want to include in MVP, the longer it will take to build. Generally, the MVP will contain only the necessary set of features needed to test the platform with initial users and stakeholders. 
• Technical complexities: Regardless of the type of AdTech or MarTech platform you want to build, there are always some key challenges that will need to be resolved, such as setting up the integrations with other AdTech, MarTech, and data platforms.

In software development, the goal is to design and build the MVP of a product within 6 months, but this largely depends on the factors listed above and also the client’s business goals and requirements. Some projects may last 6 months while others may last 18 months.  

At Clearcode, we’ve worked on many AdTech and MarTech development projects over the years. If you’d like to learn more about what’s involved in designing and building these platforms, then view our case studies below or get in contact with our team.

Case studies of AdTech and MarTech development projects we’ve worked on:

• Supply-Side Platform (SSP) Development
• AdTech Stack Development Project for an OTT Publisher
• Demand-Side Platform (DSP) Development
• Personalization and Ad Serving Platform
• Analytics Suite Development Project for Piwik PRO

What Does Renting an AdTech or MarTech Platform Mean?

Renting an AdTech or MarTech platform essentially means signing an agreement with an existing vendor and then paying a subscription or licensing fee, usually on a monthly basis. This is by far the most popular option as it allows companies to start using the platform straightaway, i.e. they don’t have to build it first.

What Does Buying an AdTech or MarTech Platform Mean?

Buying an AdTech or MarTech platform typically means acquiring an existing company and using the tech for your own purposes or adding it to your client offering. 

This is the least popular option because it typically requires a large sum of money for both the legal work (e.g. doing due diligence and all the legal costs associated with acquiring a company) and the actual purchase of the company (i.e. the AdTech or MarTech vendor). 

Although the AdTech and MarTech industries have recently seen an increase in merger and acquisition (M&A) activity, buying an existing platform is really only for the large players with a lot of available cash.

Here are some recent AdTech and MarTech acquisitions:

• Twillio acquired Segment for $3.2 billion in October 2020.
• BloomReach acquired Exponea in January 2021 (deal information undisclosed).
• AppLovin acquired MoPub from Twitter for just over $1 billion.
• TransUnion acquired Neustar for $3.1 billion.
• Magnite acquired SpotX for $1.17 billion.

Building an AdTech or MarTech Platform

What are the advantages of building an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

Below are the main advantages of building your own AdTech platform:

• Data ownership — owning the data you collect makes it easier for you and your clients to comply with various data protection and privacy laws, such as the GDPR. You’ll also eliminate the risk of your data being used by your competitors.
• Ownership of the intellectual property (IP) — not only will this give you control over the code base and algorithms, but it can also increase the value of your company.
• Control over the features and product roadmap — you can create proprietary features and algorithms to differentiate yourself from the competition. This not only ensures you get the features you want but also means the platform will be built to achieve your business’s goals, strategy and vision.
• Reduction of fees and commissions — most AdTech vendors charge a markup of 10% to 30%, which can add up if you have a large ad spend or inventory. Building your own AdTech platform can lower this amount and help you save millions of dollars per year.

What are the disadvantages of building an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

The main drawback of building your own AdTech platform is that it requires a large financial and time investment, particularly in the beginning, compared to renting an AdTech platform. It may take a minimum of 6 months to build the first working version of your platform (i.e. the MVP) but it will take much less time to get up and running with an existing AdTech or MarTech platform.

However, if your goal is to earn revenue from the platform by offering it to clients, then this isn’t so much of an issue as your goal will be to get a return on your investment over time.

You’ll also need to consider how you’ll provide the technical support needed to maintain the platform after it’s been built. Often, the development company that built the MVP for you can offer ongoing monitoring and maintenance of the platform.

Renting an AdTech or MarTech Platform

What are the advantages of renting an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

Below are the main advantages of renting an AdTech platform (i.e. using an existing platform):

• You can start running campaigns straightaway — in most cases, you can start buying or selling media within hours of creating an account.
• Lower upfront cost — a majority of AdTech platforms charge a percentage of media spend, meaning you don’t need to pay any upfront costs. This typically isn’t the case with some white-labelled solutions or larger AdTech vendors as these may charge an ongoing monthly fee on top of media spend.

What are the disadvantages of renting an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

The main disadvantages of renting an AdTech or MarTech platform are:

• No control over the features or product roadmap: When renting an AdTech or MarTech platform, you have little to no control over its current or future features. This can be problematic if you’re using the platform on behalf of your clients as you won’t be able to build new features that they request.
• No ownership of the tech or data: While most companies (e.g. agencies and brands) don’t need to own the tech or data, there are some companies whose only option is to own the tech and data, i.e. AdTech and MarTech companies. Publishers, media companies, telcos, and medium- and large-size agencies can also benefit from owning the intellectual property as it will provide them with full control of the tech, which is important when integrating it with their other systems, and also can often increase the value of their company.
• Fees and commissions: Depending on the type of company you run and how much you spend on media or earn from ad revenue, you may find that building your own AdTech or MarTech platform can help you save money in the long run, even when you take into account the cost to build and maintain it.

Buying an AdTech or MarTech Platform

What are the advantages of buying an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

The main advantages of buying an AdTech or MarTech platform are similar to those you’ll get if you build it, such as ownership of the tech, IP, and data.

You’ll also have control over the features and product roadmap, however, you may be somewhat limited in what customizations, integrations, and features you can build. It will all depend on how the platform has been built, e.g. the quality of the codebase and the infrastructure’s architecture.  

There’s also the added advantage of being able to make use of the platform immediately, instead of having to wait for it to be built. 

Buying an AdTech or MarTech platform can also provide you with a competitive advantage, such as if you acquire a platform that allows you to gain instant access to a specific industry or introduce new revenue streams.

We saw this scenario play out with telcos over the past few years. One prime example is AT&T’s acquisition of AppNexus (now Xander) to enter the AdTech industry. However, not all AdTech acquisitions have worked out for telcos.  

Other examples can be found in other industries, such as media. Recently, Roku acquired the DSP vendor dataxu to expand its advertising business.

What are the disadvantages of buying an AdTech or MarTech platform (DSP, SSP, CDP, etc)?

The main disadvantage of buying an AdTech or MarTech platform is the cost. 

There are many costs associated with acquiring a tech company, including legal fees and the actual purchasing costs. Most of the time, the cost to acquire a tech company will be many millions of dollars, but can even be in the hundreds of millions or even billions. 

For that reason, this option is typically reserved for established companies with a lot of capital behind them. 

There’s also the risk that the acquisition may not result in a positive ROI or bring additional value (in money terms) to the company. We’ve seen many examples of companies acquiring tech companies only for them to sell them off or write them down a few years later.   

In many cases, building an AdTech or MarTech platform will work out cheaper than acquiring an existing one and it can help you decrease risk by incrementally building the tech and getting feedback from users along the way to increase the value of it over time.

FAQs About Building AdTech and MarTech Platforms

How long does it take to build an AdTech or MarTech platform?

This largely depends on the software development process. 
For example, the AdTech development teams at clearcode aim to build a working minimum viable product (MVP) within 4-6 months. 

The image below illustrates our software development process when building AdTech and MarTech products:

How much does it cost to build an AdTech or MarTech platform?

Similarly to the above question, this will vary across software development companies. 

If you are evaluating different development options (e.g. an in-house development team, outsourcing company, or a specialized technical partner), it’s just as important to evaluate their experience in designing, building, and maintaining AdTech platforms as it is their proposed costs for the project. 

Working with an inexperienced development team may work out cheaper in the shorterm, but if they don’t know how the online advertising technology ecosystem works, then it might mean that you’ll spend $100,000 dollars on an MVP that doesn’t work at all or doesn’t meet the main business objectives (e.g. buy video inventory from premium websites).

AdTech

3 Key Areas Ad Agencies Need to Consider Before Renting or Building a DSP

Maciej Zawadziński, +1

May 11, 2015

AdTech

3 Ways Clearcode Can Help You with Your AdTech or MarTech Project

Mike Sweeney

Jan 04, 2018

Clearcode

AdTech/MarTech Development and the Vicious Cycle Plaguing Vendors and Agencies

Maciej Zawadziński, +1

Jul 16, 2018

The post Should I Build, Rent or Buy an AdTech or MarTech Platform [infographic] appeared first on Clearcode.

Cloud service providers like AWS allow us — software engineers and DevOps — to develop and implement new applications at a very fast pace.

Our development team recently conducted an internal project whereby we built a working minimum viable product (MVP) of an ad exchange — a piece of advertising technology that facilitates the buying and selling of online ads via real-time bidding auctions.

Ad exchanges emerged in the late 2000s with the introduction of real-time bidding (RTB). At first, they were standalone platforms that sat between demand-side platforms (DSPs) and supply-side platforms (SSPs).
 
However, over the past few years, many SSPs have incorporated exchange capabilities into their product and many ad exchanges have added features found in SSPs (e.g. inventory management and yield optimization). 

In this post, we refer to an ad exchange as a component of an SSP, but it can also operate as a standalone platform.

About Our Project

A few things to highlight:

• We only used AWS services and chose managed services when we could.
• We built the ad exchange using the least amount of code possible.
• We wanted to be able to run reports by applying any date range and attribute filters (e.g. device, domain etc.) and break them down by any dimension available (any attribute) with the option to add sub-dimensions.
• Logs are stored on S3 in parquet format.
• The data volume is 10 to 20 billion auctions per day.
• No real-time reporting is included.
• We optimized costs where possible.
• We load and report on part of the data (e.g. the current week, month), and then we archive the data to save on costs.

To explain how we achieved this, we’ve divided this article into four parts:

• Visitor: Data gathered about visitor actions.
• Processing: Based on what the visitor generated, we create a data model that is complementary to the business’s needs. All enrichments take place here.
• Tracking pixel: Collect data from the ad request.
• Post processing: Make the collected data available to external tools.

Visitor

This part, as the name suggests, relates to the visitor and the decision-making process that goes into showing them an ad when they visit a web page.

When a web page starts loading, information about the visitor (geolocation, device model, etc.) is passed to the ad exchange either by collecting it directly from the publisher’s website or via a supply-side platform (SSP).

The ad exchange then passes this information to demand-side platforms (DSPs). This is known as a bid request.

If the advertiser operating the DSP wishes to display an ad to that user, they return a bid response to the ad exchange. If their bid is the highest, their ad is displayed to the user.

Internet users may notice that ads are displayed at the same time as other pieces of content, giving the false impression that this ad-serving process is somewhat simple.

However, serving an ad to a visitor is an extremely time-sensitive process — typically, an ad exchange needs to send out the bid request to other AdTech platforms in under 50ms. 

Sending these requests to external systems (e.g. DSPs) in real time requires an extremely fast processing application. 

Here’s a detailed description of this process and how it relates to the AWS services:

Here’s an overview of what happens:

• When a user visits a web page (e.g. http://example.com, which is typically served via a CDN or load balancer), a JavaScript SDK with a configured AdUnitID is activated.
• The configured DNS record points the SDK request to the load balancer (e.g. ALB) which in turn sends a request to EC2 instances (C or R instance types are recommended).
• To speed up the ad-serving process, an ElastiCache instance is used together with in-memory cache.
• The exchange system (an EC2 instance) queries in-memory cache with a fallback to ElastiCache.
• When no AdUnitID is found, some fallback ad is sent back and displayed to the visitor.
• If AdUnitID is found, an EC2 instance executes an auction between defined vendor(s) — typically DSPs and ad networks — in a limited amount of time (usually under 250ms). This can be one vendor or multiple vendors.
• The winning vendor then sends the ad markup to the browser and displays the ad to the visitor.
• While the ad is being displayed, all information gathered about the visitor and the ad itself are passed to Kinesis for further processing.

Processing

In this part, we gather data provided by the visitor (from the bid request) and enrich it to meet various business needs. 

We first start by saving data from AWS Kinesis in S3 buckets, which is done in batches. 

Also, whenever a new file is created in this bucket, a new object creation event is triggered and launches an AWS Lambda function. This function is used to process the newly created file. 

Depending on the needs of the business, this can be one Lambda function or multiple. Regardless of the number of functions, the process is the same: take data from one bucket, process it via Lambda, and move it to another bucket. 

Every Lambda function can have its own logic and way of processing the data. 

For example, one process can query a database (e.g. RDS) for the SSP-related information and another can access an external API based on data from the visitor’s request (geolocation, device information, etc).

Finally, the processed data is stored in an S3 bucket. 

From there, the data can be queried from Amazon Athena like normal SQL queries. 

In parallel, the same data can be stored in DynamoDB tables to speed up the access required in the next step. 

Tables with configured TTL should be used to limit the availability of data to some time range (e.g. one week) in order to optimize costs.

Tracking Pixel

In this step, we verify the information acquired about the visitor relative to their real behavior, interactions, and business-related processes performed within the SSP.

When a visitor sees an ad, a pixel is fired (typically from CloudFront) and the requested data (in the URL) is stored as access logs in an S3 bucket. 

As in the previous step, we’re using Lambda functions. 

This Lambda function, however, tries to get all the information about the ad request, which is stored in the previous step. 

Impression logs are once again stored in another S3 bucket, which can then be queried from Amazon Athena.

Post Processing

The final step involves analyzing the data and making it available to external tools. 

To achieve this, we have to move all the data from Athena (in an aggregated form) to Redshift, which is great for aggregating and processing large amounts of data. 

From there, the data can be queried from an SSP interface, allowing publishers and AdOps to view reports on the SSP’s dashboard or in other data visualization tools. 

Again, we are using AWS Lambda functions, launched in one minute intervals.  

Key Takeaways

To wrap up this article, I’d like to leave you with the main takeaways:

• AWS is an ideal solution for processing large amounts of data very quickly without having to do much coding. Its managed services also greatly reduce the maintenance burden on developers.
• In our experience, we’ve found that it’s the ideal choice for building AdTech platforms as you can easily scale the application (via auto scaling), as well as bring new products to market sooner.

AdTech

How The Real-Time Bidding (RTB) Ad Exchange Works [infographic]

Mike Sweeney

Nov 30, 2015

AdTech

How Do First-Price and Second-Price Auctions Work in Online Advertising?

Maciej Zawadziński, +1

Nov 14, 2017

AdTech

The Anatomy of a Supply-Side Platform (SSP) [infographic]

Mike Sweeney, +1

Jun 12, 2019

The post How We Built an Ad Exchange with Amazon Web Services (AWS) appeared first on Clearcode.

In this blog post, we’ll explain why these universal ID solutions and ID graphs exist, how they work, and the challenges they face.

What Is a Universal ID?

A universal ID, also known as an alternative (alt) ID, is a unique user ID that allows AdTech companies to identify users across different websites and devices. Universal IDs are created using a piece of deterministic data, such as an email address or phone number. A hashed and encrypted ID is then created, allowing companies to identify the individual without exposing the raw data — i.e. email address or phone number.

Some universal IDs operate within one environment, such as web browsers, while others aim to identify users across different environments, such web browsers and mobile devices. For the latter, device graphs are used to match together the IDs generated in web browsers with the ones generated in other devices, e.g. mobile IDs in smartphones.

Universal IDs have emerged in response to the end of third-party cookies in major web browsers like Safari and Firefox, and the planned end of third-party cookies in Google Chrome in 2025.

These universal IDs perform the same functions as third-party cookies, but the difference is in how they are created.

Why Does Online Advertising Need an ID Solution?

The need for standardized ID solutions results from a number of problems AdTech faces:

1. Being able to identify users as they move around the web is vital for everyone in the online advertising industry

Publishers earn more money on their inventory, advertisers are able to achieve better campaign performance, and AdTech companies can sell their tech.

2. Web browsers don’t emit a persistent ID

Mobile devices like smartphones and tablets emit a persistent device ID – it only changes if a user manually resets it. AdTech companies can access this device ID during ad requests for in-app inventory, making it easier for advertisers to identify and target users as they move between apps.

Web browsers (on laptops, desktops and mobile devices) don’t have a persistent ID, which is why AdTech companies create third-party cookies and universal IDs to identify users on different websites.

3. Identifying users on web browsers is done via third-party cookies, but they are disappearing

The problem with third-party cookies is that they are increasingly blocked or deleted by ad-blocking software, browser settings (think Apple’s ITP and Firefox’s Enhanced Tracking Protection), manually deleting cookies, or browsing in private/incognito mode.

Also, Google Chrome has announced that it will shut off support for third-party cookies in 2025, meaning that all of the major web browsers will not support third-party cookies in near future.

4. Domains other than the one that created the cookies cannot read them

A DSP cannot read or access a cookie set by an SSP, which makes identifying the same user difficult and creates technical complexities when running behavioral and retargeting campaigns.

To solve this problem, AdTech companies use a process known as cookie syncing. This process works by matching the cookie IDs of one AdTech vendor with another.

Read more about cookie syncing here.

5. Cookie syncing is a resource-intensive and slow process

Hundreds of web calls are made with every page load, which impacts the website’s speed and the overall user experience.

6. Cookie syncing is not always reliable

The match rate of cookie IDs between different AdTech and MarTech platforms typically varies between 40–60%, and as the number of platforms involved in the syncing increases, the match rates decrease.

7. Walled gardens like Google and Meta have the advantage of using deterministic data, e.g. login data tied to people-based IDs

Programmatic vendors still rely on cookies, resulting in many advertisers and publishers turning to the walled gardens due to the advantages they offer in terms of identity, tracking and targeting.

8. Access to mobile IDs is harder on iOS devices

In June 2020, Apple made changes to how app developers can access a user’s IDFA (ID for advertisers).

Basically, app developers have to ask users to give them access to their device’s IDFA before they can pass it on to their AdTech and mobile measurement partners.

Without the IDFA, AdTech and mobile measurement companies can’t identify individual users. This means they can’t run behavioral targeting or retargeting.

The First Universal ID Solutions

Currently, dozens of universal ID solutions help solve identity issues connected with the end of third-party cookies. But the landscape of universal IDs has evolved over the years. A few years ago, there were various solutions designed to help companies reduce latency caused by the cookie-syncing process. 

Let’s take a look at them.

The Trade Desk’s Unified ID 1.0

The Trade Desk, a market-leading demand-side platform (DSP), created its Unified ID 1.0 (UID) to decrease the number of cookie syncs conducted on web pages.

Cookie syncing is necessary because every AdTech vendor uses its own identifier for users, leading to slow page loads and reduced match rates. Unified ID 1.0 aimed to standardize this process by using a single cookie ID across the open web to increase match rates and improve overall efficiency in ad targeting across web and mobile browsers.

The first iteration of this UID used the adserver.org and adsrvr.org domains to power the ID-resolution service across web browser environments (i.e., web and mobile web browsers). 

However, Unified ID 1.0 was heavily reliant on third-party cookies, which began to disappear from the AdTech landscape as the industry shifted towards greater user privacy and stricter data protection regulations.

These moves signaled the end of cookie-based tracking and forced the industry to innovate beyond cookie-reliant solutions​.

In response, The Trade Desk launched Unified ID 2.0 — the solution designed to work without third-party cookies.

The UID 2.0 creates IDs out of user-provided email addresses. This approach ensures that the identifier is persistent even when third-party cookies are blocked​. Read more about UID 2.0 in the section The Universal IDs of Today.

Advertising ID Consortium

The Advertising ID Consortium was an independent group governed by representatives from AdTech companies like Index Exchange, LiveRamp, The Trade Desk and Dataxu, and boasted the support of a number of other ad platforms. 

The group utilized cookie IDs (TTD’s Unified ID and DigiTrust’s ID) and their own cookie ID via AppNexus’s domain, as well as people-based identifiers supplied by LiveRamp’s IdentityLink (IDL). 

However, due to disagreements about the direction of the ID solution, the Consortium collapsed, and the ID project was closed down.

DigiTrust ID

Launched as a neutral, industry-wide initiative, DigiTrust aimed to standardize and simplify cookie-based user identification. It provided encrypted and standardized IDs to its members, promoting a vendor-agnostic approach. 

Despite its neutrality, the project could not survive the decline of third-party cookies and eventually DigiTrust ID was shut down in June 2020.

The Universal IDs of Today

The Trade Desk’s Unified ID 2.0 (UID2)

The Trade Desk launched the successor of the UID 1.0 in July 2020.

The  UID2 framework enables deterministic identity for advertising opportunities on the open internet for:

• Advertisers
• Publishers
• Demand-side platforms (DSPs)
• Supply-side platforms (SSPs)
• Single sign-on (SSO) providers
• Customer data platforms (CDPs)
• Consent management providers (CMPs)
• Identity providers
• Third-party data providers
• Measurement providers

Because of the growing decline of third-party cookies in popular web browsers like Safari and Firefox, and soon in Google Chrome, the second iteration replaces the use of third-party cookies with IDs created from encrypted email addresses and phone numbers, i.e., deterministic data.

Watch the video below to learn more: 

But using hashed and encrypted email addresses as an ID is only one function of Unified ID 2.0. 

During the AdWeek Spotlight Event, The Trade Desk’s CEO, Jeff Green, outlined the four main elements of Unified ID 2.0:

More than 200 entities are on the Unified ID 2.0 partners list.

Within the UID 2.0 workflow, there are a couple of key participants — the Core Administrator and Operators. These entities handle different tasks in the UID2 ecosystem.

The Core Administrator is an organization that manages the UID2 Core Service — the service accessing secured data in the UID2 ecosystem. For instance, the Core Administrator provides UID2 operators with encryption keys and salts and notifies operators and DSPs of user opt-out requests. Currently, this is held by The TradeDesk.

Operators manage the Operator Service via the UID2 APIs. This means that they obtain and retain encryption keys and salts from the UID2 Core Service, salt and hash personal data to return raw UID2s, produce UID2 tokens, and distribute decryption keys for these tokens.

The TradeDesk established public and private operators, also named open and closed operators.

Open operators run public instances of the Operator Service and deliver them to all relevant UID2 participants. However, the demand for UID2 public operators has been low, so many operators of UID 2.0 are private.

Private operators run an internal version of the UID2 service. They only use their own tech stack to process and hash first-party data — either from themselves or from clients — to create UID2 IDs.

The TradeDesk integrated with several prominent entities in this area, such as IPG, Paramount, Disney, Optable, Salesforce, Adobe, Snowflake and Amazon Web Services.

Here’s the overview of how UID2’s ecosystem works:

ID5

ID5 is an independent identity provider that allows publishers, data providers and AdTech companies to run addressable and measurable advertising campaigns.  

In July 2019, ID5 announced that its identity solution is available in Prebid.js, allowing publishers and AdTech vendors to utilize the ID in header-bidding auctions. 

The company partnered with many ad agencies, SSPs, AdTech vendors, including brands such as OpenX, The TradeDesk, and TransUnion.

ID5’s goal is to build an identity infrastructure to power addressable advertising across different channels, e.g. web/display (web browsers), in-app mobile and CTV. The company processes around 10 billion signals a day and also offers a device graph, which can be licensed, and an SDK for in-app mobile.

Compared to other universal ID solutions (e.g. Unified ID 2.0), ID5 can create an ID based on both deterministic and probabilistic data.

ID5 claims to be the most common and persistent ID present in programmatic transactions.

Secure Web Addressability Network (SWAN)

The Secure Web Addressability Network (SWAN), aka SWAN.community, is another ID solution that is very similar to UID2, but with a few differences.

Here are the main things you need to know:

• SWAN is open sourced and decentralized.
• SWAN will produce an ID, called Secure Web ID (SWID). 
• To use SWAN, publishers and consent management platforms (CMPs) will need to work with at least one SWAN Operator (more on this below).
• Users run an audit to see which companies have handled their data. 
• The main companies behind SWAN.community are Zeta Global, 51Degrees, Open X, ENGINE Media Exchange (EMX), PubMatic, Rich Audience and Sirdata.  

The SWAN Ecosystem

SWAN runs inside the SWAN Ecosystem, which is made up of SWAN Operators and SWAN Senders and Receivers.

1. SWAN Operators — companies that will facilitate the use of SWAN Data by publishers within the SWAN Network. 
2. SWAN Senders and Receivers — companies that will use the pseudonymous identifiers for identifying users and their preferences during transactions (i.e. media-buying processes). During transactions, SWAN data is sent from the Sender to the Receiver. To provide transparency, the transactions must be cryptographically signed by the Sender and the Receiver.

How Does SWAN Work?

When an Internet user visits a website using SWAN for the first time, they’ll be presented with a consent box (example below) where the user can reset their SWID, opt in to personalized advertising (optional), and provide their email address (optional). 

If a user does provide their email address, then it can be used with other ID solutions that generate IDs via hashed email addresses, such as UID2. 

Once a user has updated their preferences, the information will be passed to SWAN Operators who will then update the user’s preferences across the SWAN Ecosystem. 

The SWID will be stored in a first-party cookie and can be used for ad targeting, frequency capping, measurement, and attribution. 

Once a user has selected their preferences, they won’t be shown the consent box again on websites using SWAN. Users can also update their preferences at any time on any website participating in SWAN and will be updated across the SWAN Network. 

This SWAN initiative is not to be confused with the SWAN (Storage With Access Negotiation) proposal that was put forward by 1PlusX in the W3C Business Group as part of Google’s Privacy Sandbox initiative. 

Flashtalking Identity Management

Flashtalking, a leading ad-management and analytics-technology company, also offers a solution to the ID problem with FTrack. 

FTrack is a cookie-less tracking solution that incorporates data from different devices and across the web and mobile apps. The result is a probabilistic ID that can be used to not only target audiences, but also to attribute conversions to campaigns.

Other examples of companies offering ID resolution services include LiveIntent, and Throtle, lifesight, Navatiq, and Adara.

ID Solutions and ID Graphs

In addition to the ID solutions listed above, many of the top data platform vendors have also released their own universal ID solutions. 

The main difference between these solutions and the ones above is that they are connected to a ID graph, allowing companies — primarily, advertisers, AdTech companies and publishers — to identify individuals across different devices, rather than just across different websites.

LiveRamp

LiveRamp is a company offering many data-related services, including first-party data onboarding and ID resolution.

RampID is LiveRamp’s advanced identity resolution solution, designed to provide a persistent and privacy-compliant way to identify users across devices and channels.

RampID integrates both first-party and third-party data to create a comprehensive and persistent user identity. It uses deterministic data sources, such as email addresses and phone numbers, ensuring high accuracy and stability​.

It maps user interactions across various touchpoints, including mobile, desktop, and offline environments, allowing marketers to maintain consistent and personalized communications with their audience.

Tapad

Tapad is another global leader in the ID graph and resolution space. 

The Tapad Graph allows marketers to run cross-device ad targeting, personalization, and attribution by identifying users on an individual and household level and creating a single customer view. 

In February 2021, Tapad released a new product called Switchboard, which aims to provide interoperability to all the cookieless IDs (e.g. first-party cookies, mobile IDs, and CTV IDs) that will replace third-party cookies. 

Tapad’s Switchboard product facilitates interoperability among various identity solutions, and by using machine learning and probabilistic matching, it helps connect first-party cookies, mobile IDs, and other identifiers in a cookieless world 

Neustar

Neustar, part of TransUnion, developed TruAudience, an identity graph solution for omnichannel marketing. The product enhances media performance with next generation identity resolution capabilities.

Powered by Snowflake, TruAudience receives and integrates data from various channels, creating a single, cohesive view of each customer. It links fragmented data using a persistent ID that incorporates the richest set of historical and up-to-date attributes and identifiers for over 250 million individuals. This includes traditional identifiers like addresses, emails, and phone numbers, as well as digital identifiers such as IP addresses and Mobile Ad IDs.

TrueAudience provides a holistic view of customer interactions across all channels and touchpoints.

Epsilon

Epsilon is a digital marketing and data company that was purchased by Publicis Group in April 2019. Epsilon operates an ID graph that relies on cookies and mobile IDs, but it also has its own ID — its CORE ID.

Epsilon’s CORE ID is an identity resolution solution that enables publishers and advertisers to create a unified and detailed view of their audience. Built on deterministic data, particularly offline identifiers like name and address, CORE ID offers accuracy in mapping consumer identities across various touchpoints.

Zeotap

Zeotap is a customer intelligence platform that allows brands and publishers to connect their CRM data with IDs in Zeotap’s ID+.

The product works across both web and app properties on both mobile and desktop devices by linking a universal ID to every customer profile in Zeotap’s CDP.

ID+ is scalable and interoperable. Built on top of a deterministic identity graph, it connects with all other identifiers.

How Do ID Graphs Work?

ID graphs have been around for a number of years, but they are starting to really take off. 

The reason for the increase in popularity stems from the identity challenges mentioned above. 

Because the main identification mechanism (i.e. third-party cookies) is becoming less and less available, there’s a need to look for other ways to identify users across not only websites but also other devices like mobile and CTV. 

And that’s where ID graphs come in.

By collecting multiple IDs from different channels, ID graphs can create a single customer view (SCV), which consists of an individual’s behavior and data points across multiple devices and channels. 

Below is an example a single customer view from Piwik PRO’s customer data platform (CDP): 

Here’s an overview of how most ID graphs work:

Here’s an explanation of how ID graphs work:

Step 1. Data collection: A company would send its customer IDs (i.e. first-party IDs) to the ID graph. These first-party IDs could be taken from websites, mobile apps, and customer and data platforms (e.g. CRMs, CDPs, and DMPs). 

Step 2. Match the customer IDs with the IDs in the graph: The company’s first-party IDs would then be matched with all the other IDs in the graph, which would be done using a combination of deterministic and probabilistic matching.  

Step 3. Activate the data for cross-device activities: The company can now identify their customers across different devices and channels and run various cross-device activities, like ad targeting, personalization, and attribution. 

The Challenges Facing These ID Solutions

Apart from the fact that some of the solutions don’t scale in the same way as they once did — i.e. IDs created from hashed email addresses aren’t as readily available as IDs stored in third-party cookies — all these identity solutions face the same challenges; they still revolve around identification and rely on some type of ID. 

This is a problem because companies with walled gardens, like Google and Apple, are constantly strengthening their products to make them more privacy-friendly. 

We’ve seen this with Apple’s ITP and changes to IDFA, and even Google has made changes to how Chrome handles third-party cookies and is also planning on phasing them out soon (possibly in 2025). 

AdTech companies are moving from one identification method to another, and it won’t be long until Apple and/or Google make some changes to their web browsers or mobile operating systems to prevent these identification methods. 

For this reason, many folks in the industry believe that these ID solutions are rather short-term solutions. 

Many say that the future of digital advertising and marketing won’t be done on an individual basis but rather done in a privacy-friendly way where individuals are not identified. 

It’s too early to say if and when identification will disappear completely, so in the meantime, companies need to use an identity solution like the ones listed above to ensure they can still run effective advertising and marketing campaigns. 

Data & Privacy

What Is a Data Clean Room and How Does It Work?

Mike Sweeney, +1

Apr 07, 2022

Data & Privacy

What Does the EU’s DSA and DMA Mean for AdTech?

Mike Sweeney, +1

Oct 10, 2022

Data & Privacy

What’s the Difference Between Zero-Party, First-Party and Third-Party Data?

Natalia Figas

Mar 15, 2023

The post Identity in AdTech: Meet The Various Universal ID Solutions [Updated 2024] appeared first on Clearcode.

Indeed, the Wild West is famous for lawlessness, vigilantism, and outlaws on the run. 

Are there any similarities to how the online advertising industry operates today?

Authorities Slow to Act

The Wild West saw an influx of new land – and later, gold – for the taking, which happened at such a fast rate that authorities were slow to introduce laws to regulate and control it. 

The introduction of the public internet in the mid-1990s created a similar situation, but instead of land and gold, user data was the prize. 

Since the mid-2000s, online advertising-technology companies have been collecting, segmenting and sharing huge amounts of consumer data, and just like in the Wild West era, authorities and governments have been slow to act.

It’s not like advertising companies have done all of this without any safeguards; concepts like privacy by design and data minimization have been around for a number of years. 

This has brought us to the current state where companies have amassed user data to the point where it’s become the backbone of the online advertising industry, and where user-data collection happens on any device with an internet connection (just look at how many tags and pixels are fired on some websites), and is passed to multiple vendors via piggybacking and cookie syncing.

However, several events over the past few years suggest that these days are numbered. 

The EU’s General Data-Protection Regulation (GDPR) kicked things off, with other countries like Brazil and India looking to introduce similar regulation. 

Even large tech companies like Cisco, Microsoft and Apple are pushing for a GDPR-style law in the US, not to mention the lingering proposed ePrivacy regulation, which will likely have a bigger and more direct impact on AdTech than the GDPR has had. 

Despite being stuck between negotiations on the text of the draft and the all-important trilogue discussions, it would be wise to think about how it will impact AdTech and the changes needed to comply with it now.

Over the coming years, AdTech companies will need to ensure their data-collection activities are compliant with various data-protection and privacy laws – or face the wrath from authorities.

Vigilantism in the Form of Self-Regulation

Violence and theft were rife in the days of the Wild West, with local authorities unable to control the sheer number of crimes due to lack of manpower. 

In response, vigilante justice groups were formed by members of the community to play judge, jury and executioner, essentially doing what the authorities couldn’t. 

In the online advertising world, there’s a sense of vigilantism in the form of self-regulation. 

Governments and other authorities have simply lacked the resources needed to control and regulate the sprawling online advertising ecosystem, leaving the industry to form self-regulatory organizations such as the Interactive Advertising Bureau (IAB) and the Network Advertising Initiative (NAI), among others. 

However, unlike the vigilante justice groups of the Wild West, the self-regulatory groups in online advertising have done little to properly regulate and manage the out-of-control data-sharing activities conducted by AdTech companies. This is despite attempts to do so with failed initiatives such as AdChoices (which requires users to opt out by clicking on tiny icon found on ads, and then wait as a new page loads the user’s browser and cookie statuses) and the IAB’s Transparency and Consent Framework.

Time’s Up for Shady Actors

The infamous outlaws and bandits of the Wild West would hide out in the badlands, away from the prying eyes of authorities and vigilantes. 

For the better part of a decade, advertising companies and data brokers have collected and distributed user data, all without the user’s knowledge or explicit consent, but the online-advertising bandits can hide no longer.

Consumers are more aware of the extent of data collection by walled gardens (Google, Facebook, etc.) and independent AdTech and data companies, thanks in part to complaints filed by privacy groups, such as the recent ones by Brave and the Open Rights Group. These filings formed the basis for a separate complaint filed by Panoptykon as well.

A recent report by Piwik PRO highlighted that 74% of websites placed third-party cookies without valid consent and 86% of websites preloaded possible tracking cookies even before the user expressed their consent decision. The third-party cookies almost always belong to AdTech or MarTech companies, meaning personal data is being passed outside the EU, exposing them to retargeting, profiling or even sale of their data.

Successful AdTech companies of the future will be the ones that respect user privacy; both clients and consumers will simply not support companies that don’t.

This means the onus is on AdTech and data companies to prove to both clients and consumers that their data-collection practices are privacy-friendly and in compliance with the GDPR (and whatever other data-protection and privacy laws may follow). 

The Events That Ended the Wild West

It’s hard to pinpoint one event that ended the Wild West era, but it was likely a combination of new laws that regulated land ownership and gold mining, as well as the takedown of many prominent outlaw figures.

With all that has happened in the past few years (ad blockers, GDPR, and ITP), could the Wild West days of mass user-data collection in online advertising be coming to an end?

The evidence for this is mounting. 

While the introduction of the GDPR didn’t have an immediate impact and clearly wasn’t the catalyst for change for a large majority of AdTech companies, the recent investigations by French Data-Protection Authority CNIL and the UK’s Information Commissioner’s Office (ICO) into Google’s violation of the GDPR regarding user consent could be a clear sign of things to come for all AdTech companies.

The next few years will likely establish law and order regarding user-data collection, regulate what self-regulation couldn’t, and expose the AdTech companies that fail to adhere to data-protection and privacy rules.

Now is the time to act to make AdTech more privacy-friendly, reduce the amount of user data companies collect, and regain trust of online users to ensure our business models remain viable for the future.

Fifty years from now, the AdTech world of today may be seen as a chaotic, user data free-for-all – just don’t expect any motion pictures to be made about it.

This article was originally posted on Clearcode’s Medium publication.

Data & Privacy

Don’t Fret – the GDPR Is a Good Thing for AdTech

Maciej Zawadziński, +1

Dec 08, 2017

Data & Privacy

GDPR and ePrivacy FAQ from AdTech vendors

Maciej Zawadziński, +1

Apr 19, 2018

Data & Privacy

What Does the EU’s DSA and DMA Mean for AdTech?

Mike Sweeney, +1

Oct 10, 2022

The post Taming the Wild West of Consumer Data Sharing in AdTech appeared first on Clearcode.

To solve this problem, a new type of AdTech platform emerged — network optimizers. Then, when real-time bidding (RTB) was introduced in the late 2000s, these network optimizers evolved into supply-side platforms (SSPs).

Let’s now take a closer look at the components and features that make up many of today’s supply-side platforms.

Click here to open this SSP infographic in a new tab.

What Is a Supply-Side Platform (SSP)?

A supply-side platform (SSP) is an advertising technology (AdTech) platform that helps publishers monetize their websites and mobile apps by managing, selling and optimizing their available inventory (aka ad space). SSPs are a key player in real-time bidding (RTB) media transactions, whereby publishers sell display, video and native ad space to advertisers on an impression-by-impression basis.

Components of a Supply-Side Platform

Backend and Infrastructure

In order for an SSP to provide its features to publishers and sell ad space, the various backend components need to be hosted on an infrastructure (e.g. Amazon Web Services). From there, they can carry out all the various technical processes that power the SSP’s features.

Integrations

SSPs integrate with other AdTech platforms to facilitate the selling of inventory (for example with ad servers, DSPs, and ad exchanges) and with data-management platforms (DMPs) to maximize their revenue.

Read our post about how to integrate a DSP with an SSP/ad exchange.

Ad Exchange

An ad exchange orchestrates the buying and selling of ads between advertisers and publishers. Many SSPs now incorporate ad-exchange functionalities, meaning publishers can directly connect (via an SSP) to advertisers (via DSPs), rather than having to first connect to external ad exchanges.

Trackers

Trackers collect data about the publisher’s website and its audience. It then sends this data to other components, such as the user-profile database and reporting database.

Reporting Database

The reporting database receives campaign and audience data from the tracker, which allows publishers to generate reports and view campaign analytics (see Analytics and Reporting below).

Features of a Supply-Side Platform

User Interface

The user interface is the screen that publishers use to manage campaigns, view reports, manage billing and use other features of the SSP.

Analytics and Reporting

Once data has been sent from the reporting database, publishers can create and view reports about the performance of their inventory, including fill rates, clicks and impressions.

Header Bidding

Many SSPs incorporate header-bidding functionality, allowing publishers to obtain bids from multiple demand sources (e.g. DSPs) before their ad server is called. The header-bidding feature of an SSP enables publishers to manage their various header-bidding wrappers and demand partners.

Read our previous post about header bidding to learn more.

Yield Optimization

The yield-optimization feature of a supply-side platform aims to increase revenue for publishers by improving fill rates, setting floor prices and managing auction mechanics (e.g. first- and second-price auctions).

Inventory and Campaign Management

This feature allows publishers to manage different types of inventory (display, video, native, etc.), blacklist and whitelist advertisers, set IAB categories and block certain types of ads.

AdTech

How We Built an Ad Exchange with Amazon Web Services (AWS)

Mike Sweeney, +1

Sep 26, 2019

AdTech

Top Demand-Side Platform (DSP) Companies [Updated in 2024]

Mike Sweeney, +1

Sep 30, 2020

AdTech

Top Supply-Side Platform (SSP) and Ad Exchange Companies [Updated in 2024]

Paulina Zawiślak, +1

Jan 19, 2021

The post The Anatomy of a Supply-Side Platform (SSP) [infographic] appeared first on Clearcode.

As it turned out, the world didn’t end. Computers still worked and digital calendars displayed the correct date, apart from a few somewhat-minor incidents.

In the lead up to the GDPR’s enforcement date on May 25, 2018, the online advertising world experienced a similar hysteria.

Just like the Y2K bug, however, the AdTech world didn’t collapse, companies didn’t file for bankruptcy, and ads were still being bought and sold programmatically or otherwise.

In fact, it almost seems the opposite is true when you consider some of the news that’s come out of the digital advertising industry since the GDPR came into force almost a year ago.

Stock Prices Are Up and Advertising Activity Is in Full Swing

One impact of the GDPR that nobody predicted was an increase in stock prices of some of AdTech’s largest publicly traded companies.

The Trade Desk has seen its share price climb from $82.99 on May 24, 2018, to $202.09 at the time of publishing this post, as well as healthy growth in terms of revenue.

Similarly, Rubicon Project’s stock price was $2.34 on the day the GDPR kicked in, and now sits at $5.76 a share (again, at the time of writing).

There was also AT&T’s monster purchase of AppNexus for close to $2 billion, with similar acquisitions likely to occur as large telecommunications companies look to compete with the duopoly of Google and Facebook.

On the outside, it certainly looks as if the AdTech industry is in a pretty healthy state.

However, the above companies are based in the US, with TTD experiencing significant growth in non-display areas, like connected TV and audio.

When you look at AdTech companies in Europe operating mainly on display advertising, it’s a different story. Criteo, for example, has seen its share price fall from $25.35 on May 24, 2018, to $19.07 on May 29, 2019.

Unlike Y2K, the potential negative effects of the GDPR aren’t immediate, and the worst could still be yet to come, meaning the positive stock growth and large acquisitions likely won’t last in the long term.

Here’s why:

Incorrectly Collecting User Consent

Although the GDPR doesn’t spell out in black and white what consent must look like, it certainly provides enough details for companies to understand what their user-consent mechanisms should and shouldn’t include.

For example, the regulation clearly states that consent must be freely given and data controllers (like publishers) can’t deny service or access to users if they refuse to provide consent or if they don’t state their preferences.

Despite this, many publishers are denying access to users if they don’t agree to hand over their data. Some have adopted “assumed consent,” meaning anything other than a direct “no” (closing the consent form and continuing to use the site without selecting their preferences) constitutes consent.

These are in addition to the widespread practice of firing third-party tags when a user rejects to their data being processed. While there’s a need to collect and store a user’s consent decisions, and therefore must be passed along to a publisher’s AdTech partners, there’s a big chance the user’s data will still be passed (or “leaked”) to various AdTech platforms.

Considering these non-compliant tactics, it’s no surprise that we’ve seen companies boast opt-in rates of 90%.

These methods of obtaining user consent are not GDPR-compliant and go against one of the main purposes of the regulation—to give data subjects more control over their information.

Legitimate Interest Incorrectly Used as Legal Basis for Data Processing

Even before the GDPR rolled into existence, parties from all sides of the AdTech ecosystem had placed all their chips on legitimate interest as their legal basis for processing personal data.

While it doesn’t state in so many terms that data processing for advertising and marketing isn’t an example of legitimate consent, this argument will likely crumble if put before an EU court.

It seems that the only way companies can collect and process personal data is by obtaining user consent, which, as mentioned above, still isn’t happening correctly.

Companies Still Operating as ‘Business as Usual’

Apart from the recent increase in some AdTech stocks, another surprise to come out of the AdTech world was news that retargeting had actually grown in the months following the GDPR’s implementation.

It seems that many advertisers are moving away from prospecting, which heavily relies on third-party data collected from a multitude of sources, and are moving towards retargeting, as it’s considered a safer option.

While certain areas of online advertising, such as the size of available audiences, has been impacted for the most part, it looks like it’s business as usual for a majority of the folks out there—with legitimate interest and non-compliant user-consent mechanisms helping keep the entrance doors open.

The Full Impact Hasn’t Arrived… Yet

As we approach the one year anniversary of the GDPR, it still seems that it hasn’t impacted the AdTech industry the way many expected, but it’s still in the early days.

Google received a $57 million fine from the French data protection agency, CNIL, for its lack of GDPR compliance. We also saw CNIL investigate Vectaury, a French mobile ad location DSP, for its non-compliant collection of user data. However, CNIL later withdrew the investigation in February 2019, stating that the AdTech vendor no collects valid consent.

These stories have no doubt made a lot of AdTech players take notice, but haven’t been enough to bring about any significant change in the way user data is collected.

Privacy groups and advocates are helping bring to light many of these non-compliant practices, which is causing various data protection agencies to act.

My prediction is that the move towards true GDPR compliance will be a slow one that will take a few years, spurred on by public awareness and legal proceedings.

We avoided the Y2K disaster because companies invested heavily in preparing for its arrival. Most companies haven’t made the same investment with the GDPR. The AdTech doomsday that many thought would happen never materialized, but it’s still too early to rule it out.

Data & Privacy

On-Demand Webinar: The Impact of the GDPR on the AdTech and MarTech Industries

Mike Sweeney, +2

Mar 20, 2019

Data & Privacy

A Timeline of GDPR Events in AdTech & MarTech

Mike Sweeney, +1

May 25, 2020

Data & Privacy

What Does the EU’s DSA and DMA Mean for AdTech?

Mike Sweeney, +1

Oct 10, 2022

The post Why the GDPR Hasn’t Impacted AdTech… Yet appeared first on Clearcode.