One of the things you’ll learn a lot about if you pursue any sort of formal education in computing science is data structures. However, once I hit the real world then it seemed like for most problems the speed differential between a linked list and a System.Collections.Generic.List wasn’t meaningful. List quickly became my go-to and it was very unusual that any other data structure would even cross my mind. This sort of lazy thinking can have a serious effect as the size of input grows. So harkening back to my school days I wanted to talk about the rudimentary data structure that is a linked list.

Read More

Storage Queues are one of the original pieces of Azure dating back about a decade now. They are great for deferring work to later or spreading it out over a bunch of consumers. If you’re following best practices for DevOps you’ll know that the creation of your queues should be done in code. In some cases you can create the queues on application startup but in serverless scenarios there often is no startup code so the responsibility of creating queues falls to your deployment process. Let’s look at how to do that on Azure DevOps

Read More

Durable Functions provide a way to run a series of functions in an orchestrated concert*. The product is relatively new and it is difficult to find a whole lot of guidance on how to do things with it. I found logging to be an area where some guidance was needed. So I’m writing some.

Read More

Need to set the version of Azure functions runtime from an ARM template? Weirdly it is actually an app setting and not something on the site. The variable is called FUNCTIONS_EXTENSION_VERSION and can take on the values ~1 or ~2. In an ARM template it looks like

{
    "type": "Microsoft.Web/sites",
    "kind": "functionapp",
    "name": "[variables('webSiteName')]",
    "tags": {
        "displayName": "Site"
    },
    "apiVersion": "2016-08-01",
    "location": "[resourceGroup().location]",
    "scale": null,
    "resources": [
    {
        "name": "appsettings",
        "type": "config",
        "apiVersion": "2015-08-01",
        "dependsOn": [ "[concat('Microsoft.Web/sites/', variables('webSiteName'))]" ],
        "tags": {
        "displayName": "App settings"
        },
        "properties": {
        "FUNCTIONS_EXTENSION_VERSION": "~2",
        }
    }
    ],...
}

Azure is huge. There are probably a dozen ways to host a website, a similar number of different data storage technologies, tools for identity, scaling, DDoS protection - you name it Azure has it. With that many services it isn’t unusual for me to find some service I didn’t even know existed. Today that service is Data Factory. Data factory is a batch based Extract, Transform and Load(ETL) service which means that it moves data between locations. I mention that it is batch to distinguish it from services which are online and process events as they come in. Data Factory might be used to move data between a production database and the test system or between two data sources.

Read More

If there is one thing that we developers are good at it is holy wars. Vi vs. Emacs, tabs vs. spaces, Python vs. R, the list goes on. I’m usually smart enough to not get involved in such low brow exchanges… haha, who am I kidding? (vi, spaces and R, BTW) Recently I’ve been tilting at the windmill that is checking in package files. I don’t mean the files that tell what version of files to check in but the actual library files.

Read More

Two of the major ideas de jour in development circles these past few years have been DevOps and Microservices. That they rose to the forefront at the same time was not a coincidence. They are inexorably linked ideas.

Read More

Just the other day somebody was mentioning to me that they were having trouble setting up a statically hosted site on AWS. That was the kick in the nose I needed to get this article written as it’s been on my back-burner for a while. Terraform makes the whole process easy.

Read More

There seems to be very scant examples on how to download a file from S3 using node and JavaScript. Here’s how I ended up doing it using the aws-sdk:

const params = {
    Bucket: 'some bucket',
    Key: 'somefilename.txt'
  };

const s3Data = await s3.getObject(params).promise();
console.log('Content length is:' s3Data.ContentLength);
console.log('Content type is:' s3Data.ContentType);
console.log('Writing to file');
const file = fs.createWriteStream('c:/afile.txt');
file.write(s3Data.Body);

I ran into an error today creating an SQS queue in serverless which looked a lot like:

CloudFormation - CREATE_FAILED - AWS::SQS::Queue - SendEmailQueue
...
An error occurred: SendEmailQueue - API: sqs:CreateQueue Access to the resource https://sqs.us-east-1.amazonaws.com/ is denied..

You would think this error is related to something in IAM, and indeed it may well be. However in my case it wasn’t it was actually that I was giving CloudFormation a mal-formed name for the queue. I would like to think that the developers of SQS would use HTTP code 400 (bad request) to report this error but instead they use 403. So keep in mind that even though it looks like a permissions problem it might be a syntax error.