I’ve been working on some client applications that call on Lambda to do different things. In one case, the Lambda function calls an API. In the second instance, it accesses a database. In both cases, I was getting weird errors that we hard to trace down.

In the case of the database, I was returning more data than I’d asked for. Lambda was returning data from the previous query, as well as that from the current query. In the case of the API call, I had errors when I first ran it. But I fixed the error, could see the ‘good’ data, but I was still returning to the client an error message.

It took days to figure it out in the first case. I knew some sort of cacheing was going on, but I couldn’t figure out in which layer. So I put in print statements in client. Yup, the duplicates were there (obviously – that’s where I first observed them). I checked the API Gateway. Yup. Duplicates there. I checked Lambda. Dups there. I did the same query using the MongoDB interface, Compass, and received no dups. So I went back to Lambda.

Seems it was where I was instantiating the class calling the database. I was doing it outside the ‘handleRequest’ method. Why? No particular reason. Since the class had one method, I didn’t think it would make a great deal of difference. But apparently it does.

What I’m guessing is that the Lambda engine instantiates the class once for whatever period of time, and just calls the method every time an event is received. So, basically, the data I’d had returned from the database was cached in my global variable and just appended with each query.

So I changed the database client instantiation to appear inside the method, and I also made sure to close the connection at the end of the method. This is probably not the most efficient, since it does look like Lambda is creating a connection pool which, of course is also destroyed when I close the connection. I’ll need to find a better way of clearing the old results, I guess.

But it did work. No more replicated results.

I just ran into the same problem with my API caller. But, in this case, it was the errors that were accumulating. It was the same cause: I was instantiating the error array outside of the method. So I moved the instantiation inside the method and my problems disappeared.

Having this little bit of new knowledge, I will need to examine my code and put into place some best practices: what should be a global variable? what should be local? is there a way to ‘clean up’ in between that keeps the code more efficient?

Topics for later!