Centralized AWS logging with OpenSearch, Cognito, and Terraform
Cross-account logs into a single OpenSearch domain, Cognito in front of the dashboards, the whole thing in Terraform.
On Datakeeper the production logs did not live in one account. Each client account had its own services, and troubleshooting meant guessing which console to open. Centralizing that was the useful part. The cluster itself is not the interesting problem.
This note is the first half of that setup: the OpenSearch domain, and Cognito so people can sign in without me handing out AWS keys. The CloudWatch subscription filters that actually ship the logs are the second half. The Terraform lives in Warns/opensearch-centralized-logging.
01One place to look.
A multi-account platform is normal once you have more than one production tenant. It is also a bad place to go hunting for a stack trace. I wanted one OpenSearch domain, fed from CloudWatch in the other accounts, with a login that was not “borrow my IAM user.”
OpenSearch is the search and dashboards half. Cognito is the front door: a user pool, groups, MFA, and an identity pool so the dashboards can assume a role. Terraform is how it gets repeated, instead of clicked together once and forgotten.
02A public domain, a locked door.
The domain in opensearch.tf is a public endpoint, not in a VPC. Access is Cognito plus MFA, not network position. Zone awareness is on so a bad availability zone is not an outage. Encryption at rest and node-to-node encryption are on. Fine-grained access control takes a master user ARN from an IAM role, not a username baked into the domain.
cluster_config {
instance_count = var.instance_count
instance_type = var.instance_type
zone_awareness_enabled = var.zone_awareness_enabled
dynamic "zone_awareness_config" {
for_each = var.availability_zone_count > 1 ? [true] : []
content {
availability_zone_count = var.availability_zone_count
}
}
}
cognito_options {
enabled = true
user_pool_id = aws_cognito_user_pool.pool[0].id
identity_pool_id = aws_cognito_identity_pool.identity_pool[0].id
role_arn = aws_iam_role.cognito_role[0].arn
}EBS size, instance type, warm nodes, and dedicated masters are variables. The shape of the cluster should not be a commit every time someone wants a bigger disk.
03Two kinds of user.
cognito.tf builds the user pool, turns on MFA, and then splits people into two groups. Masters get AmazonOpenSearchServiceFullAccess. Limited users get read-only. Both are created from a list of emails, with a generated temporary password sent by email, and dropped into the matching group.
resource "aws_cognito_user_group" "master_user_group" {
count = module.this.enabled ? 1 : 0
name = var.master_user_group_name
precedence = 1
role_arn = aws_iam_role.master_user_role[0].arn
user_pool_id = aws_cognito_user_pool.pool[0].id
}The identity pool refuses unauthenticated identities. Role mapping resolves an ambiguous token by denying it, so a half-configured login does not land in a useful role. There is also a user-pool domain and an app client, which is where this got fiddly.
04OpenSearch creates a client for you.
A normal Cognito app client is aws_cognito_user_pool_client. That is enough when nothing else is involved. OpenSearch is involved. When the domain is pointed at Cognito, the OpenSearch service creates its own user-pool client. Terraform will not see that client unless you adopt it with aws_cognito_managed_user_pool_client.
That resource is only for the client OpenSearch made. Don’t use it for a client you created yourself. The identity-pool role mapping has to point at whichever id is actually there: the managed client if OpenSearch has created it, otherwise the one Terraform made. This was a provider gap for a while; the fix landed in hashicorp/terraform-provider-aws#30140.
resource "aws_cognito_managed_user_pool_client" "aws_identity_pool" {
count = module.this.enabled ? 1 : 0
name_prefix = "AmazonOpenSearchService-${module.this.id}"
user_pool_id = aws_cognito_user_pool.pool[0].id
depends_on = [aws_opensearch_domain.default]
}05Logs are not in the cluster yet.
What this gives you is a secure domain and a front door: full users, read-only users, MFA, no anonymous dashboards. It does not yet move a log line. That is CloudWatch subscription filters in the source accounts, a Lambda on the logging account, and a destination the filters are allowed to call.
Same modules, different accounts. The domain stays here. The filters get repeated wherever the services run. I used this pattern for the ten Datakeeper production accounts that needed one place to look.