DATA Pill #069 - is ELT dead? Chatbot with LLMs, DevOpsGPT

ARTICLES

The Netflix team demonstratehow they combined multiple machine learning models used in Netflix's large-scale search and recommendation systems into a single unified model, simplifying the architecture, improving performance and enabling faster system development. They share the trade-offs and lessons learned from this approach for broader applications.

Let’s discuss the evolution of data processing architectures, from ETL to ELT and finally to the current EtLT architecture. This text explores the reasons behind these changes, their strengths and weaknesses and why EtLT is emerging as the dominant data processing architecture, along with open-source implementations like Apache SeaTunnel to meet modern data infrastructure demands.

In today's e-commerce landscape, exceptional customer service is not a choice but a must. Online shopping's growth has increased the need for personalized experiences and 24/7 support. This blog will show how to build an efficient e-commerce assistant with Large Language Models (LLMs), highlighting their development complexities and capabilities.

This article explores the differences between Apache Flink and Kafka Streams in terms of their capabilities, use cases, scalability and fault tolerance. It also discusses the learning curve and resources required for both frameworks, ultimately concluding that the choice between them depends on the specific needs of the application, with Apache Flink being more generalized and Kafka Streams more specific to stream processing.

How to advance and democratize artificial intelligence through open source and open science? What makes Falcon 180B so good? This one looks at some evaluation results and shows how you can use the model.

Ericsson successfully moved 80% of its applications to the cloud under CIO Mats Hultin and VP Johan Sporre Lennberg's leadership. This transformation enhanced agility and innovation, fostering cultural alignment between IT and the business, while streamlining operations and facilitating rapid adoption of technologies like AI.

Let’s introduce entity-centric data modeling (ECM), a novel approach that prioritizes "entities" (such as users, products and campaigns) at the forefront of analytics, by merging ideas from dimensional modeling and feature engineering to enhance data representation.

This one offers insights into Pokémon GO's infrastructure scaling with Google Cloud services like Spanner and Kubernetes for handling large user requests. It describes the request flow, involving components like CDN, NGINX, game services, Bigtable storage and Pub/Sub for analysis.

This text explores various secure integration methods, such as Azure Machine Learning with Prompt Flow, Power Apps, Snowflake External Function and Snowpark External Access, as well as Streamlit's role in enabling interaction with Azure OpenAI for Data Apps on the Snowflake Data Cloud.

TUTORIAL

This article is compiled from Xiaolin He, Alibaba's Senior Technical Expert and Apache Flink PMC Member and Committer, who shared it at the 2022 Flink Forward Asia (FFA) Conference. This article is mainly divided into three parts:

1. Flink SQL Insight
2. Best Practices
3. Future Works

NEWS

In this report, the analysis involved examining usage data from over 20,000 customers across various major cloud platforms, where they monitored their serverless workloads with the platform. The report presents essential insights into how these customers utilize serverless technologies in practical situations.

TOOLS

Welcome to the AI Driven Software Development Automation Solution, abbreviated as DevOpsGPT. It combines LLMs (Large Language Models) with DevOps tools to convert natural language requirements into working software. This innovative feature greatly improves development efficiency, shortens development cycles and reduces communication costs, resulting in higher-quality software delivery.

This blog explores kcctl, a new open source command line tool for Kafka Connect. You'll find out how to integrate it with Apache Kafka and manage connections to other systems.

DATA TUBE

Dr. Andrew Ng leads a discussion on AI's potential and impact, emphasizing the significance of supervised and generative AI tools, the rise of low-code and no-code AI development, untapped opportunities across industries and the importance of responsible AI for addressing challenges like pandemics and climate change, while dispelling exaggerated fears of AI causing human extinction.

CONFS EVENTS AND MEETUPS

Cut through the clutter, harness generative AI's potential for your industry. Join innovative engineers and leaders, master generative systems, build better models, find cost-effective infrastructure, and gain a strong support network for faster production.

Agenda & topics covered

• Breaking through the noise: how your organisation can innovate
• Quantifying uncertainty in generated models to create more reliable products
• Powering your GANs & VAEs with state-of-the-art compute for rapid output
• A fully monetised generative AI landscape: how to drive revenue in a new ecosystem

…and more.