AWS.NZ

Fix Codex resume after switching AWS accounts on Bedrock

Sun, 25 Oct 2026 13:30:00 +1300

I switched the AWS account used for Codex inference on Amazon Bedrock. An old chat still opened, but its next turn failed.

The workaround was to remove the old account’s encrypted state from the saved chat. The original messages and tool results stayed in place. Codex could then replay that history without sending the old encrypted data.

Observed outcome: a local test confirmed that the repaired chat produced a request with no encrypted content. I applied the repair with a backup. Live Bedrock inference was not tested during this investigation.

Fine-tune Llama-2 with Amazon SageMaker JumpStart

Mon, 04 Dec 2023 19:48:49 +1300

Customising Llama-2 for your task offers a cheaper and more secure LLM experience.

Image by author and DALL-E 3

The advancements in Large Language Models over the past few months unleashed some interesting opportunities for text processing automation. Let’s take a look, for example, at removing Personally Identifiable Information (PII) from free-form text data. Have a look at this call centre log (made-up example):

October 10, 2023, 2:15 PM, customer Maria Gonzalez called in to inquire about her mortgage loan payments. Agent Brian Clark confirmed the customer’s identity using a verification code sent to their mobile number 027-456-7890 and by confirming her home address as 45 Wellington St, Christchurch, 8010. […]

Bitcoin mining on AWS in 2023 — the complete guide

Thu, 02 Nov 2023 21:46:27 +1300

The last time I wrote about crypto mining on AWS we were still before “The Merge” — the (in)famous 2022 event when Ethereum switched from Proof of Work to Proof of Stake which spelled the end of ETH mining. The lucky people with free AWS accounts then switched to mining Ethereum Classic (ETC) or Ethereum POW (ETHW) but these were very unprofitable, yielding perhaps 10% or 20% max on a good day. And that’s not 20% profit, that’s getting $20 in crypto for $100 spent in AWS costs. Not a good deal.

Crypto mining on AWS and GCP after “The Merge”

Tue, 06 Sep 2022 13:36:41 +1200

Yes, you can still mine crypto in the public clouds even after Ethereum switch to Proof of Stake!

Ethereum mining and “The Merge”

For a long time Ethereum (ETH) used to be the favourite (= most profitable) crypto coin to mine on GPUs. People were building “mining rigs” — specialised computers equipped with lots and lots of NVidia or AMD Radeon GPUs — and operated them 24/7 in order to earn new ETH coins. The cost of setting up these rigs was massive, many thousands or even tens of thousands of dollars. And this upfront cost was obviously a big barrier for many wanna-be miners.

Setting up Fargate for ECS Exec

Tue, 16 Aug 2022 10:25:04 +1200

Examples for both CloudFormation and Terraform.

Photo by Jason Leung on Unsplash

In the previous post I have introduced a new convenient ecs-session tool for logging into your Fargate ECS containers through the ECS Exec mechanism. Today we’re going to explore the ECS configuration needed for ecs-session to work.

ECS Exec requirements

The containers that you want to access via ECS Exec / ecs-session must meet the following requirements:

Easy Ethereum mining on GCP

Fri, 05 Aug 2022 19:14:48 +1200

ETH mining will be gone in a month or two when the Ethereum network migrates to the long anticipated Proof-of-Stake model. While now it no longer makes sense to buy GPUs and build mining rigs it’s still perfectly possible to mine on the Google Cloud Platform.

Photo by Kanchanara on Unsplash

In the previous posts I have introduced an easy way to mine Ethereum (ETH) crypto currency on the AWS platform. Many readers asked me whether the same can be done on GCP, the popular Google Cloud Platform. Of course it can be done! GCP also offers VM instances with NVidia GPU cards — namely Tesla T4, Tesla V100 and the super fast Tesla A100 — and we can use them for ETH mining.

Login to Fargate containers the easy way

Thu, 07 Jul 2022 14:14:02 +1200

Discover ecs-session — the new easy to use tool for interactive access to ECS tasks

Photo by moren hsu on Unsplash

Running apps in Docker containers comes with some challenges. Most notably we are not supposed to “login” to the containers and should only observe them from the outside. However let’s be honest — sometimes it may be handy to login to a running container and “do something”. For example we may want to check the network connectivity from the container point of view. Or to test access to the AWS services with the container’s IAM role credentials. Or to check some logs that we forgot to forward to CloudWatch. And so on. I’m not saying that we should be doing it on daily basis, but sometimes it’s required…

Mining Ethereum on AWS — the complete guide

Wed, 01 Jun 2022 22:36:42 +1200

Ethereum mining on AWS is now even easier with a ready to use template. Try it today because ETH mining may be over soon!

Photo by Kanchanara on Unsplash

The AWS Ethereum Miner project has been around for over a year now and has seen a great interest from the miners during this time. With the looming end of Ethereum mining — aka ETH 2.0 or “the merge” (see below) — it no longer makes sense to invest in buying GPUs and mining rigs. There simply isn’t enough time to pay them off.

Create intelligent CloudFormation templates with Custom Resources

Sun, 24 Apr 2022 18:25:10 +1200

When I wrote my first CloudFormation template to facilitate Ethereum mining on AWS it was a relatively simple solution used only by me. I could handle the little inefficiencies here and there. But then I made it public and people started using it. And they ran into issues.

“The template doesn’t work in region x-y-z!” Oh…
“The template doesn’t work with instance type abc.xlarge!” Oh…
“I want to use both spot and on-demand!” Oh…
“Why is it starting g5.xlarge when g4ad.2xlarge is more efficient?” Oh…

To please the crowd I had to improve the template and make it more intelligent.

The new AWS instance that makes ETH mining profitable

Sun, 02 Jan 2022 12:38:00 +1300

Mining Ethereum on AWS until now wasn’t profitable. This new AWS instance is the game changer that no only breaks even but can finally make you money!

Ever since I wrote my first article about Ethereum and Bitcoin mining on AWS more than a year ago I was bombarded over and over again with the same sort of questions :

Is it profitable now that the ETH price has increased?
… that this new AWS instance type was released?
… that this new coin is the hype of the day?

So far the answer was always a NO. The profitability has gradually improved from some $20 mined in ETH for every $100 spent on AWS services a year ago, to $60 or $65 in ETH for the same $100 AWS spent a month ago. You would still be losing money though.

Mining Ethereum in AWS — is it worth it?

Mon, 06 Dec 2021 21:42:04 +1300

At around this time last year I wrote my first article about Mining Bitcoin and Ethereum on AWS and figured out that back then it wasn’t worth it. Over the past year the ETH price has increased from $450 to $4500 — let’s find out whether mining on AWS is finally profitable!

Photo by Executium on Unsplash

A lot has happened since I wrote my previous article about mining on AWS:

Amazon Rekognition support for ZoneMinder object detection

Fri, 10 Sep 2021 17:38:40 +1200

ZoneMinder is a popular open-source video surveillance system. AWS Rekognition is an easy to use image recognition service from Amazon. Together they are a perfect match.

Photo by Bernard Hermant on Unsplash

ZoneMinder is a remarkably versatile system. It’s compatible with pretty much any IP camera that you can buy, you can run it on your own server and therefore you don’t have to trust your security video feeds to some random vendor with dubious security standards. Being open-source there are no monthly “subscription fees” nor any expensive optional “power packs”. It’s all free.

Mining Bitcoin and Ethereum on AWS

Fri, 18 Dec 2020 21:23:28 +1300

The Bitcoin mania has struck again. With the BTC and ETH prices at all times high many people look for ways to take part in the frenzy. Start mining on AWS today using our easy to use CloudFormation template. Or better not? Let’s find out…

Photo by André François McKenzie on Unsplash

Typically you can obtain Bitcoins and other crypto-currencies in one of three ways: by buying them, trading them or mining them. Here we will have a look at the latter option — mining crypto on AWS.

Recover lost access to your EC2 instance

Thu, 06 Aug 2020 17:06:42 +1200

I’m being asked the same question over and over again:

I got locked out of my EC2 instance! I did “something” and now I can’t SSH to it. How do I fix that?

Photo by Jonathan Rados on Unsplash

Usually this happens during server hardening or some other “fixing the server” attempts gone wrong. From turning on the iptables firewall without leaving port 22 open, through changing some file or directory permissions, to right out messing up the SSH daemon configuration.

Analyse PDFs at Scale with Amazon Textract — Part 2

Tue, 26 May 2020 11:11:45 +1200

In the previous post, I described my need to parse many PDF documents in an automated way. We have introduced Amazon Textract — a service that can do that at scale with surprising accuracy. Then we have gone through the architecture and limitations. Today we will deploy the solution, parse some documents and use the results.

Photo by Austin Neill on Unsplash

Create the stack and test

The complete Textract CloudFormation project is available on GitHub. Follow the instructions there to deploy the stack to one of the regions where Textract is available. For example to us-west-2 (Oregon) which also happens to have the lowest Textract prices.

Analyse PDFs at Scale with Amazon Textract — Part 1

Tue, 26 May 2020 10:39:50 +1200

Photo by The New York Public Library on Unsplash

It’s a Tax time again here in New Zealand. When I was putting together the documents for my accountant last week, I wanted to export some account statements from my bank to CSV.

Surprisingly I realised that it could be done for most of my accounts except for one — the mortgage loan account. The bank helpline confirmed that CSV export is not possible, and the only option is to download the monthly statement PDFs and re-type them to Excel manually. Wait! What? Why? Come on! It works for all my other accounts, why not for the mortgage one?!

CloudFormation Service Roles

Sat, 10 Aug 2019 20:45:33 +1200

Sometimes we want to give users the ability to create pretty much anything with CloudFormation but at the same time prevent them from doing the same through the console or aws-cli. Perhaps it’s a company policy that everything must be managed using CloudFormation. Or on the other hand you may have Admin privileges but want to CloudFormation can do on your behalf.

Solution?

CloudFormation Service Role

CFN Service Role can achieve exactly that – users can have a separate set of credentials for work outside of CloudFormation and a completely different set of credentials for creating, updating and deleting CloudFormation stacks.

ssm-tunnel

Sun, 21 Jul 2019 23:22:51 +1200

EC2 instances, RDS database and many other resources reside in VPCs. To access them we need a network connectivity – either through a VPN, Direct Connect, or through a Jump / Bastion Host. That’s stating the obvious.

Sometimes, however, it’s inconvenient or difficult to achieve a direct connectivity ad-hoc – maybe you are in a location where your VPN is filtered, or from where the Bastion host Security Group won’t allow access but you still need access to the RDS for example.

SSM Sessions the easy way

Mon, 27 May 2019 08:05:08 +1200

In the previous post – Using SSM Session Manager for interactive instance access – I showed you how to access EC2 instances through AWS Systems Manager (SSM) Sessions without having to open Security Groups or firewall ports, maintain SSH keys, VPNs, Jump Hosts, and so on.

The native way for starting a SSM Session with aws ssm start-session requires the knowledge of the exact Instance ID, e.g. i-01234567890abcdef. That’s not very user friendly unfortunately. I know what my “pet” instances Host Names or Instance Names are. And I can often see the IP addresses of a misbehaving auto-scaling instances in the logs. However I don’t always know the corresponding Instance ID.

Using SSM Session Manager for interactive instance access

Sat, 25 May 2019 00:41:17 +1200

You may argue that interactive login to EC2 instances should never be needed. Everything is dynamic, automated, self healing, centrally logged, and so on and there is no place for human interaction, right? But lets be honest – the world isn’t perfect and we all sometimes need to jump into bash to do stuff for one reason or another.

Traditionally our EC2 instances would have a SSH KeyPair assigned and a Security Group with SSH port open. If they were on private IPs only we would also need a jump host or VPN to access them. That works but it’s got its own problems – for example rotation of the SSH keys, keeping the Security Group up to date, etc.

Amazon Polly: Teach your projects talk

Fri, 29 Mar 2019 05:30:47 +1300

Photo by Jeff Sheldon on Unsplash

Voice output can take your project to a whole new level. Are you a maker building a new home automation tool or a professional developer working on a commercial gadget? Follow this tutorial to learn how to add a natural voice to your project with very little effort!

TTS = Text To Speech

Text To Speech systems are nothing new. The first time I heard a computer speak was sometime in the last century on my Commodore C64. It was a hardly intelligible, monotonic, robotic voice. Yet so exciting!!

AWS Polly — credentials

Wed, 13 Mar 2019 16:37:40 +1300

This is a side-story for the main article about AWS Polly.

Photo by Jose Fontano on Unsplash

AWS Account

Before we can use any AWS services we need an AWS account. If you don’t already have one Create a Free AWS Account now. The account creation requires a credit card however as long as you stay within the Free Tier you won’t be charged anything.

AWS Polly User

Once we have an account open we will need the credentials for our script. Explaining the AWS Security model is well beyond the scope of this article, suffice to say that we’ll create a so called IAM User with an attached IAM Policy that gives it access to the AWS Polly service.

CloudFormation package & deploy

Wed, 20 Feb 2019 20:51:08 +1300

Easily deploy complex CloudFormation templates with external resources such as Lambdas or Nested Stacks.

Many CloudFormation templates are completely standalone – one single YAML or JSON file and that’s it. Easy to deploy. However in some cases CFN templates refer to other files, or artifacts. For example Lambda source or ZIP file, nested CloudFormation Template file, or an API definition for API Gateway may be such “artifacts”. These files have to be available in S3 before we can deploy the main CloudFormation template.

IP-Ranges Updater

Fri, 28 Dec 2018 17:16:18 +1300

Back in March I wrote about a command line script filter-ip-ranges that can parse the Amazon-published ip-ranges.json file and look up address ranges by region, service or IP address. That’s handy for the occasional manual use or for creating or updating some whitelists or Security Groups through aws-cli.

But running an aws-cli script from cron on some EC2 instance is so old school!

IP-Ranges-Updater Lambda

Much better way is to have this functionality in a Lambda function and let it update your Security Groups or Route Tables automatically.

New C5 / M5 / T3 instance disk names

Tue, 02 Oct 2018 21:07:44 +1300

When we started using the new C5, M5 or T3-class instances (e.g. t3.micro, m5.large, m5d.large, c5.xlarge, etc) we noticed that the disk device names are no longer the classic /dev/sda1 or /dev/xvda that we were used to for years.

TL;DR

The new root disk name is /dev/nvme0n1 with the partition device names being /dev/nvme0n1p1, /dev/nvme0n1p2, etc.

… just in case you’re lazy to read the whole post 🙂

Identifying the disks

Now that the disk names in the OS are no longer the names shown in the web console how do we know which disk is which? That’s even more pressing issue if you launch an M5D instance (e.g. m5d.large) with instance ephemeral disk.

CloudFormation Exports

Mon, 17 Sep 2018 22:26:01 +1200

If you are a CloudFormation user you have probably come across a situation where an ID of a resource created in one stack (e.g. Subnet ID from VPC-Stack) is needed in another, independent stack, for example to create an EC2 instance in EC2-Stack.

The traditional approach is to pass Outputs from one template to Parameters in another template. However there are a few problems…

The first problem with this method is that is that you somehow have to collect the Outputs from one stack and pass them to the Parameters in another. This can be done either manually or through some scripting like AWS-CLI or Ansible, but it’s an extra, mundane work.
The second problem is that many stacks need the same set of parameters – VPC ID, Subnet IDs, perhaps some Security Group IDs, Route53 hosted zone name and ID, etc. The Parameters list can get very long and not very interesting.

There is a better option – CloudFormation Stack Exports

CloudWatch Log Mailer

Sat, 30 Jun 2018 21:34:06 +1200

Feeding logs from various AWS services to CloudWatch Logs is a common pattern. From Lambda logs, through ECS Task logs, to Linux /var/log/* messages forwarded by awslogs daemon, they can end up in CloudWatch. How can we monitor them – and get alerted – when something of interest gets logged?

People often use Elastic Search, Splunk, Graylog or similar heavy weight solutions. But do we really need that? Very often all we want is to know when a certain keyword, e.g. “Error”, appears in the messages. Luckily that’s very easy to do.

filter-ip-ranges

Sat, 31 Mar 2018 19:45:34 +1300

Sometimes it’s handy to know what IP address ranges are used in different AWS Regions. You may want to whitelist access to your region AWS endpoints, or from global CloudFront location, or set up some special routing rules, or look up which region a certain IP belongs to.

ip-ranges.json & filter-ip-ranges

AWS publishes ip-ranges.json file with all that info. But it’s a huge JSON and not really meant for human consumption. So here I wrote a small parser: filter-ip-ranges

Cross-account access with aws-cli

Wed, 10 Jan 2018 13:34:27 +1300

In the previous post I explained how to Use Cross-account access through AWS Console. Today I’ll show you how to do the same in the command line using aws-cli.

We’ve got Access and Secret keys for the Login account and want to use aws-cli to create and manage resources in the Dev account above.

First step is to configure aws-cli with credentials for the aws-nz-login account.

Using Cross-account access (AWS Console)

Sun, 10 Dec 2017 10:45:25 +1300

Short post to demonstrate how to use AWS Cross-account access from AWS Console. To re-iterate this is what we what we have set up in our previous post:

First step is to login to the aws-nz-login account as an non-privileged IAM user, e.g. as michael.ludvig in my case.

Select Switch Role

In the top-right menu under your IAM username select Switch Role link.

Dev account Role details

Fill in the Dev account name and Role

Cross-account access

Wed, 06 Dec 2017 14:50:01 +1300

In bigger organisations it is common to have one central AWS account with IAM User accounts and a whole lot of independent per-project or per-team accounts that are only through cross-account access from this central account.

The benefits is obvious – company has a single place where they manage user accounts, credentials, passwords, permissions, etc. It is a very similar concept to switching from local Linux accounts to a centralised LDAP or AD authentication.

Linux user authentication with SSSD / LDAP

Tue, 17 Oct 2017 19:42:16 +1300

Current Linux distributions can seamlessly work as members of Active Directory domains which gives them access to the AD authentication system. However it requires the Linux hosts to “join” the AD domain, for which one has to posses some special AD privileges. In many cases this is not viable and we may only want a simple user authentication without any write privileges to the Active Directory. Good old LDAP provides such an option.

Let’s Encrypt certbot-auto problems on Amazon Linux

Sun, 08 Oct 2017 14:37:03 +1300

Let’s Encrypt certbot-auto support for Amazon Linux is still marked as experimental and as such we occasionally encounter unexpected problems. The most recent one was introduced with certbot-auto 0.19.0. During install or upgrade from previous versions you may get an error like this:


Upgrading certbot-auto 0.18.2 to 0.19.0...
Replacing certbot-auto...
Creating virtual environment...
Installing Python packages...
Installation succeeded.
Traceback (most recent call last):
 File "/opt/eff.org/certbot/venv/bin/letsencrypt", line 7, in

It took me quite a while to figure out that it’s got nothing to do with Zope.

Using AWS Storage Gateway Virtual Tape Library in Linux

Wed, 20 Sep 2017 17:31:49 +1200

If you are of around my age or younger you probably didn’t have much exposure to tape backup technologies. Tapes are sooo 90’s right?! I definitely didn’t expect that as an AWS Consultant I will have to learn about tapes. But I did! One of our customers wanted to use AWS Storage Gateway (SGW) in the Virtual Tape Library (VTL) mode and use it for backups with Veeam Backup suite. Veeam seems to “just work” with SGW VTL but I like to understand how things work under the hood so I decided to backup my Linux test system to VTL using just the low level Linux tools. That’s the best way to learn.

Monitoring temperature with Raspberry Pi and AWS IoT

Tue, 25 Jul 2017 23:48:27 +1200

Working as a Senior AWS Consultant for one of the New Zealand’s leading AWS consulting providers means that I’m expected to have a hands on experience with as many AWS services as possible. In this series I will walk you through my first project that makes use of AWS IoT service.

Background

In one of our offices we’ve got a small server room with a handful of old servers – mostly sandpit stuff, nothing mission critical. The rack is cooled by an old unreliable Air Conditioning unit that sometimes stops cooling and sometimes even leaks water. Not good but not painful enough to warrant an installation of a new commercial grade cooling.

Joining a Linux EC2 instance to Active Directory realm

Tue, 06 Jun 2017 15:34:38 +1200

This post is not too AWS-specific, in fact the steps below should work not only on Amazon Linux but also on RedHat Linux, CentOS and Oracle Linux and posibly on Debian and Ubuntu based distros as well.

There is a number of prerequisities for a successful completion of this task.

At least the following records should be resolvable:

S3 Pre-signed URL example

Wed, 25 Jan 2017 19:13:17 +1300

S3 pre-signed URLs can be used to provide temporary third-party access to private objects in S3 buckets. For example, non-public files on a file sharing site can be made available to approved users with one-off URLs that expire after 10 minutes. Here we offer a simple demo for testing the concept.

The demo consists of these parts:

An S3 bucket with a private access policy, i.e. non-public.
An EC2 instance with an IAM role attached.
An IAM policy that gives the EC2 instance access to the S3 bucket through its role.
Some files, images, etc. uploaded to the S3 bucket.
A simple PHP script running on the EC2 instance that lists the objects in the bucket and generates signed URLs.

Preparation

Spin up an Amazon Linux EC2 instance.

Re-using EC2 SSH Key Pair in multiple AWS regions

Mon, 04 Jul 2016 17:33:14 +1200

Photo by Samantha Lam on Unsplash

One of the parameters required for launching an EC2 instance is a Key Pair which is effectively an SSH Key used for interactive logging into the default user account — on Amazon Linux it’s the ec2-user account — or for decrypting the Windows Administrator’s password.

It is easy to create a new Key Pair / SSH Key as part of the EC2 launch process however as soon as you start using more regions and more accounts you will quickly end up with heaps of different Keys and unless you are diligent with their naming both on the filesystem and in AWS you will end up with a mess. Like I used to.

Using LastPass with AWS Console

Sun, 17 Jan 2016 01:02:56 +1300

As an Amazon Web Services professional working on projects for multiple customers I need an easy way to login to their many AWS accounts through the web console.

Since I use LastPass for storing my website passwords it’s only logical to use it for AWS logins as well. The problem is that there are 3 login fields – Account, User Name and Password – instead of the usual 2. LastPass has support for filling the additional field however it may not be entirely obvious. Follow these steps…

EC2 Instance Roles

Fri, 27 Nov 2015 13:15:39 +1300

In the previous post about Access & Secret Keys I emphasized that those keys – static keys – should never be used in production and in fact they should never leave your laptop. But what if you need to call the AWS services from scripts or programs running on an EC2 instance?

In a typical scenario such a script is called either by a scheduler (cron) or by another program like CI/CD tool (Jenkins) to automatically perform a task that needs certain AWS privileges. Say it has to upload a newly build package to an S3 bucket at the final stage of Jenkins run or it shut down all non-production EC2 instances in the evening and restart them in the morning.
The answer is – assign IAM Role to the EC2 Instance

Michael Ludvig

Mon, 23 Nov 2015 01:17:26 +1300

Who am I?

Michael Ludvig — Experienced IT Professional focused on Architecting, Designing and Implementation of Amazon Web Services solutions for customers of various sized and various AWS adoption levels.

My background in all things Linux with over 15 years of professional experience from Linux Kernel development through System, Security and Network Administration to a more high level role of Design and Delivery of new projects makes a perfect foundation for the varied and complex skill set required for successful Amazon Web Services adoption.

Access & Secret Keys

Wed, 22 Jul 2015 00:14:16 +1200

When working with AWS API the clients (programs, scripts, etc) must have a way to prove who they are and what level of access to Amazon services they should have.

In a typical scenario an AWS user runs aws cli (or a script using aws cli) to interact with Amazon. For example to create a VPC and Subnets or to backup some files to S3 bucket. To perform any non-anonymous operation like these the script will need a set of AWS Credentials – also known as Access Key and Secret Key.