Jitesh has played a crucial part in the success of this project. His enthusiasm, dedication, communication, ownership, and technical abilities are simply mind blowing. I don’t believe that I ever worked with someone for a period as short as 10 days that impressed me as much as Jitesh did. A true star of the show.
Solving Delta Table Concurrency Issues
Solving Delta Table Concurrency Issues Delta Lake is a powerful technology for bringing ACID transactions to your data lakes. It allows multiple operations to be performed on a dataset concurrently. However, dealing with concurrent operations can sometimes be tricky and may lead to issues such as ConcurrentAppendException, ConcurrentDeleteReadException, and ConcurrentDeleteDeleteException. In this blog post, we will explore why these issues occur and how to handle them effectively using a Python function, and how to avoid them with table design and using isolation levels and write conflicts.