DATA Pill feed

DATA Pill #188 – Happy New Year! Testing Pitfalls, Streaming at Scale, Agent Design & AI Careers for 2026

ARTICLES

The Seven Deadly Sins of Test Automation|Allegro Tech | Małgorzata Kozłowska | 8 min | Testing & QA
A sharp breakdown of the most common test automation anti-patterns: fixed sleeps, brittle selectors, shared state, flaky tests and false confidence in coverage. The article also shows how to “redeem” each sin with state-based waits, isolation and better observability.
A pragmatic look at where coding assistants actually help — and where they silently introduce technical debt. The article argues that senior engineers benefit most, while teams need guardrails to avoid cargo-cult AI-generated code.
How Uber Indexes Streaming Data with Pull-Based Ingestion in OpenSearch| Uber Engineering | Yupeng Fu, Varun Bharadwaj, Shuyi Zhang, Xu Xiong, Michael Froh | 9 min | Streaming & Search
Uber explains why traditional push‑based indexing can’t keep up with the company’s real‑time scale: clients have to handle backpressure, can’t prioritise critical updates and struggle with data replay. To improve reliability, they contributed a native pull‑based ingestion framework to the OpenSearch project. The new IngestionPlugin and StreamPoller components let OpenSearch consume from Kafka or Kinesis streams at its own pace, buffering spikes and simplifying failovers
Agent Engineering: System Designs| Data Science Collective | Minhajul Hoque | 10 min | Agentic Systems
A clear decision framework for single-agent vs multi-agent systems. Covers coordination overhead, tool contention, failure modes and when multi-agent designs hurt more than they help.
A Realistic Roadmap to Start an AI Career in 2026| Towards Data Science | Leonie Monigatti | 8 min | AI Careers
Sabrine Bendimerad notes that AI education is oversaturated with bootcamps and YouTube courses. She urges aspiring practitioners to build expertise through end‑to‑end projects instead of chasing certificates: start with advanced machine‑learning problems on realistic datasets; focus on feature engineering and model interpretation; and progressively tackle MLOps, deployment and privacy. The article lays out a four‑phase roadmap that emphasises practical skills and holistic problem‑solving rather than shallow surveys

NEWS

Roblox details how its AI moderation systems process billions of text, voice and avatar interactions per day, combining real-time models with human review to enforce safety at platform scale.
OpenAI uses automated red-team attackers to continuously probe and harden Atlas, reflecting a broader shift toward adversarial testing as a core AI safety practice.

TOOLS

DeepEval, created by Confident AI, lets teams build reliable evaluation pipelines for AI systems. It integrates with PyTest for unit‑testing LLMs, offers 50+ research‑backed metrics (including G‑Eval and deterministic metrics) and supports single‑ and multi‑turn evaluations, multimodal inputs (text, images, audio) and synthetic test‑data generation.

DATATUBE

From Vibe Coding to Vibe Engineering| Kitze & Sizzy | 34 min | AI Engineer
This video explores how code‑generation tools like Sizzy and new “vibe engineering” paradigms are reshaping developer workflows. Creator Kitze discusses lessons learned from early coding assistants and demonstrates building richer, context‑aware tools that orchestrate not just code snippets but entire developer experiences.

CONFS, EVENTS, WEBINARS & MEETUPS

Confluent’s annual predictions webinar examines new technical realities of AI in 2026 and offers guidance on future‑proofing data ecosystems. Speakers will highlight why traditional databases can’t handle the query surge and present architectures optimised for speed, scale and resilience. The talk is aimed at CTOs, data architects and platform engineers and is based on insights from Confluent’s 2026 Predictions Report
Promising AI Use Cases for the Enterprise in 2026| DATAVERSITY | Webinar | January 2026
This free webinar, hosted by data‑management expert William McKnight, surveys the most promising AI applications across industries, from customer service and healthcare to travel, compliance and cybersecurity. It emphasises that early adopters gain exponential benefits by embedding AI into core business processes, and provides examples of chatbots, predictive maintenance, fraud detection, personalised medicine and more

PINNACLE PICKS

Your last edition top picks:
[POLISH ONLY] Zbuduj Agenta AI na Bielik LLM – Pydantic AI Tutorial | Wojtek Mikołajczyk, Marcin Zabłocki | ML‑Workout | 20 min
This Polish‑language tutorial shows how to build an AI agent using the Bielik large language model and Pydantic for data validation. It walks through designing prompts, structuring agent loops and deploying the agent in an end‑to‑end workflow, making it ideal for developers interested in local LLMsConcurrent is a free desktop application that lets you benchmark over a thousand AI models across 30+ providers and local runtimes. You type a prompt and the tool sends it to multiple models, then displays responses side‑by‑side with metrics for quality, speed (tokens per second) and cost.

We Spent 2 Years Building a Data Mesh — It Was a $4M Disaster | Medium | Amįń | 9 min | Data Leadership
In this candid postmortem, a data engineer recounts how a two‑year, $4 million data‑mesh initiative turned into 47 competing fiefdoms. Despite following best practices and even consulting Zhamak Dehghani, the team ended up with worse data quality, unused “data products” and missed deadlines; they are now abandoning the mesh in favor of a centralized platform

From Python 3.8 to 3.10: Our Journey Through a Memory Leak | Jay Patel| Lyft Engineering | 8 min | Data Science
Upgrading Lyft’s services from Python 3.8 to 3.10 exposed a memory leak triggered by reference cycles and caching in long‑running processes. Jay Patel recounts how engineers used tracemalloc and objgraph to identify the leak, discovered it was tied to lazy initialisation and caches, and eliminated it by explicitly clearing caches.
_____________________
Have any interesting content to share in the DATA Pill newsletter?
Made on
Tilda