Tag Archives: Lambda

Lambda and Cacheing

I’ve been working on some client applications that call on Lambda to do different things. In one case, the Lambda function calls an API. In the second instance, it accesses a database. In both cases, I was getting weird errors that we hard to trace down.

In the case of the database, I was returning more data than I’d asked for. Lambda was returning data from the previous query, as well as that from the current query. In the case of the API call, I had errors when I first ran it. But I fixed the error, could see the ‘good’ data, but I was still returning to the client an error message.

It took days to figure it out in the first case. I knew some sort of cacheing was going on, but I couldn’t figure out in which layer. So I put in print statements in client. Yup, the duplicates were there (obviously – that’s where I first observed them). I checked the API Gateway. Yup. Duplicates there. I checked Lambda. Dups there. I did the same query using the MongoDB interface, Compass, and received no dups. So I went back to Lambda.

Seems it was where I was instantiating the class calling the database. I was doing it outside the ‘handleRequest’ method. Why? No particular reason. Since the class had one method, I didn’t think it would make a great deal of difference. But apparently it does.

What I’m guessing is that the Lambda engine instantiates the class once for whatever period of time, and just calls the method every time an event is received. So, basically, the data I’d had returned from the database was cached in my global variable and just appended with each query.

So I changed the database client instantiation to appear inside the method, and I also made sure to close the connection at the end of the method. This is probably not the most efficient, since it does look like Lambda is creating a connection pool which, of course is also destroyed when I close the connection. I’ll need to find a better way of clearing the old results, I guess.

But it did work. No more replicated results.

I just ran into the same problem with my API caller. But, in this case, it was the errors that were accumulating. It was the same cause: I was instantiating the error array outside of the method. So I moved the instantiation inside the method and my problems disappeared.

Having this little bit of new knowledge, I will need to examine my code and put into place some best practices: what should be a global variable? what should be local? is there a way to ‘clean up’ in between that keeps the code more efficient?

Topics for later!

Calling AWS Lambda from PHP

Amazon Web Services (AWS) documentation is voluminous. Some of it’s good; some of it’s bad. And sometimes you have to look carefully to figure out that what you’re viewing is outdated.

Thus it was when I tried to figure out how to call a Lambda service from PHP.

I had created a web service in Java that was being deployed to AWS Lambda. We had connected it to an AWS API Gateway, figuring that a web service call to the API Gateway would be the easiest way to implement a client. Not so. Not in PHP at least.

I got the service working so that I could call it from Postman. But, when I tried calling it using PHP’s cURL library, I found that trying to calculate the signature required to pass the AWS Secret Key was a headache on good days, and would drive you to drink on bad. I can’t tell you why what I tried didn’t work. I only know that I kept getting Missing Authorization errors that I couldn’t overcome and still stay sane.

So I switched to using the AWS PHP library which, I guess, I should have used in the first place. But the documentation required numerous gotos that made it confusing to follow. There were no simple examples. There were no straightforward “here’s how it should look” instructions to follow. Eventually, however, I put the pieces together. And, hopefully, you can, too.

Starting with one example, I first worked out how to get the LambdaClient object I needed.

The first task was to get the credential information. Those who have developed for AWS on their own machines may have noticed that this information is stored in an .aws subdirectory under the user’s folders. The common way of retrieving the key and secret assume access to this subdirectory. The second way this is done assumes system variables with the values. My implementation could not make use of this method. So AWS allows for custom credential objects, which is what I used:

public function getCredentials() {
   return function () {
      // Get the credential variables
      $key = AWS_KEY;
      $secret = AWS_SECRET;
      if ($key && $secret) {
         write_log('credentials created');
         return Promise\promise_for(
            new Credentials($key, $secret)
         );
      }

      $msg = 'Could not find credential variables. ';
      return new RejectedPromise(new CredentialsException($msg));
   };
}

Since this is part of a WordPress site, the AWS_ variables are stored in the wp_config file.

The documentation recommends starting with an instance of the Aws\Sdk, then creating the client from there:

$sdk = new Aws\Sdk([
   'region' => REGION,
   'version' => 'latest',
   'credentials' => $this->getCredentials()
]);
$client = $sdk->createLambda();

… where $this->getCredentials() uses the function above.

Odd thing, though, is even after creating the client included passing credentials and location information, that was needed again when we actually made the call to the server. So I set up another function to hold that information:

public function config(){
   $config = array(
      'credentials' => array(
         'key'    => AWS_KEY,
         'secret' => AWS_SECRET,
      ),
      'region'  => AWS_REGION,
      'version' => "2015-03-31"
   );

   write_log('lambda config: ' . json_encode($config));
   return $config;
}

Yes, I know I was inconsistent with the versions. But, for this, I wanted to make sure I was using what I had programmed to, to ensure nothing broke later.

Last step was the actual call to Lambda:

public function sendMessage() {
   $client         = $this->getLambdaClient();
   $json_config = json_encode($this->config());

   $result = $client->invoke([
      'ClientContext' => base64_encode($json_config),
      'InvocationType' => 'RequestResponse',
      'FunctionName' => FUNCTION_NAME,
      'LogType' => 'Tail',
      'Payload' => json_encode($my_input)
   ]);

   write_log( $result);
   $log_unencode = base64_decode($result->get('LogResult'));
   write_log("unencoded log: " . $log_unencode);
   write_log("returned payload: " . $result->get('Payload'));

   return $result->get('Payload');
}

Interesting thing was needing to encode the configuration. Makes sense, I guess, since it’s sending login information. But that wasn’t required when creating the client object.

Also note the fact that the Payload needed to be in JSON format to be sent.

The result object that I wrote to the log was interesting to see. It didn’t include the Payload, which I thought was weird. So at first, not knowing how to get the results, I tried pulling the Cloudwatch log back to see whether I could get the results that way, since I was printing them in the log. Interestingly enough, the log was base64 encoded and had to be decoded to read.

But after printing the $result->get(‘Payload’)), I was able to find the results of the call.

This is a slow call. I’m assuming that’s because this is an ‘on demand’ service. But we will be making several of these at a time so I’m hoping the successive ones will be faster.