Senior Data Engineer (AWS)

About this role

<p><strong><em><span data-contrast="auto">Capnexus</span></em></strong><span data-contrast="auto"> is a comprehensive services provider. Our team consists of outstanding professionals, highly experienced in designing, building, and supporting retail software. We see ourselves as a build-as-a-service provider who follows a repeatable business pattern that can be applied to a variety of platforms and verticals. Having a culture built on outcomes and delivery at the core of the business, Capnexus is providing its customers with a complete suite of services for software development, system analysis, integration, implementation, and support, as well as the option to engage a single team to perform all the services they require.</span><span data-ccp-props="{"335559739":120}"> </span></p> <p><strong><span data-contrast="auto">Who You Are and What You'll Do:</span></strong><span data-ccp-props="{"335559738":120,"335559739":60}"> </span></p> <p><strong><em><span data-contrast="auto">Capnexus</span></em></strong><span data-contrast="auto"> is looking for a highly skilled </span><strong><span data-contrast="auto">Senior AWS Data Engineer</span></strong><span data-contrast="auto"> to lead data architecture, pipeline development, and data integrations. This is an exciting opportunity to apply advanced cloud data engineering skills on a platform that leverages generative AI to automate and modernize enterprise workflows.</span><span data-ccp-props="{"335559739":120}"> </span></p> <hr class="border-border-200 border-t-0.5 my-3 mx-1.5"> <p class="font-claude-response-body break-words whitespace-normal"><strong>Responsibilities:</strong></p> <ul class="[li_&]:mb-0 [li_&]:mt-1 [li_&]:gap-1 [&:not(:last-child)_ul]:pb-1 [&:not(:last-child)_ol]:pb-1 list-disc flex flex-col gap-1 pl-8 mb-3"> <li class="font-claude-response-body whitespace-normal break-words pl-2">Participate in data discovery workshops to inventory source systems including property management platforms, marketing channels, and CRM data, and translate findings into data lake architecture requirements.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Design and implement a multi-zone enterprise data lake on Amazon S3 (raw, conformed, enriched, aggregated) with ingest, cleansing, and business layers including schema versioning, checksum validation, business rule validation, and quarantine/notify workflows on failure.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Build batch and streaming data ingestion pipelines using AWS Glue, Amazon Kinesis, and containerized ingestion applications across CDP, marketing, and property management data sources.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Write PySpark and Python ETL code for AWS Glue jobs to transform, cleanse, and enrich data at scale; apply Apache Iceberg table format for ACID-compliant, schema-evolving data lake tables.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Implement data transformation and orchestration frameworks using AWS Glue ETL and AWS Step Functions; configure AWS Glue Data Catalog with crawlers for automated metadata management and discovery.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Implement AWS Lake Formation for fine-grained data governance including table-level and column-level permissions, data filters, and resource links — not just IAM-level access controls.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Configure Amazon Athena for serverless SQL querying across the data lake with performance optimization (Parquet format, partitioning, column pruning, file size management, caching); implement Amazon DynamoDB for sub-second customer profile lookups, with DAX where latency requirements demand it.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Develop and deploy AWS Lambda functions using AWS Lambda Powertools for structured logging, handler routing, and observability; implement error handling patterns including exponential backoff, retries, dead-letter queues, and CloudWatch alarms.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Write and maintain Terraform (or CloudFormation/CDK) modules to provision and deploy AWS data infrastructure as part of the CI/CD pipeline — data engineers own their infrastructure deployment, not DevOps.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Integrate CI/CD pipelines using GitHub Actions for automated deployment of Glue jobs, Lambda functions, and Step Functions workflows with lint checks and validation gates.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Support Azure Data Lake migration: conduct discovery of ADLS assets, schemas, and transformation logic; provision AWS target environments; execute migration via AWS DataSync; perform row-count reconciliation, schema validation, and checksum comparison post-migration.</li> <li class="font-claude-response-body whitespace-normal break-words pl-2">Design and implement entity…