Connection Pooling and Intermittent Failures in K8s

<h1>Intro</h1> <p>One of the aims of this blog is to occasionally publish smaller pieces of content that cover specific problems and troubleshooting situations that we encounter during our own day-to-day work. We believe this will be useful because it can a) help anyone facing the same problem save a bit of time, and b) over time teach our readers investigation methods that can be applied to other similar problems or scenarios.</p> <p>We have not yet found a good name for this type of content, so in the absence of a better option, we&rsquo;ll refer to them as &ldquo;bitesize knowledge&rdquo;.</p> <p>In this post for instance, we look at the journey of troubleshooting a real-life problem involving intermittent inter-pod connection failures in our EKS setup.</p> <h1>So what happened?</h1> <p>Recently, a developer on my team reported intermittent failures with a specific HTTP request flow that involves our services running in AWS EKS. The failures occurred when certain REST requests hit an internal API service, which then called other microservices to gather information and return a response. As the requests were synchronous, any issues with subsequent requests propagated to the client as an overall failure.</p> <p><a href="https://medium.com/binome/connection-pooling-and-intermittent-failures-in-k8s-9d1c89fbb50a"><strong>Learn More</strong></a></p>