Skip to main content

Logging with Redaction

Use Microsoft.Extensions.Logging with automatic sensitive field redaction.

Overview​

When logging DynamoDB operations, you may need to redact sensitive data like PII, financial information, or authentication tokens. FluentDynamoDB integrates with Microsoft.Extensions.Logging and supports automatic redaction of marked fields.

Configuration​

Basic Logging Setup​

using Microsoft.Extensions.Logging;
using Oproto.FluentDynamoDb;

var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.AddConsole();
    builder.SetMinimumLevel(LogLevel.Debug);
});

var options = new FluentDynamoDbOptions()
    .WithLogging(loggerFactory);

var table = new UserTable(dynamoDbClient, "users", options);

Dependency Injection​

services.AddSingleton<ILoggerFactory, LoggerFactory>();
services.AddSingleton(sp =>
{
    var loggerFactory = sp.GetRequiredService<ILoggerFactory>();
    var options = new FluentDynamoDbOptions()
        .WithLogging(loggerFactory);
    
    return new UserTable(dynamoDbClient, "users", options);
});

Marking Sensitive Fields​

Use the [Sensitive] attribute to mark fields that should be redacted in logs:

[DynamoDbTable("users")]
public partial class User
{
    [PartitionKey]
    [DynamoDbAttribute("pk")]
    public string Pk { get; set; } = string.Empty;

    [SortKey]
    [DynamoDbAttribute("sk")]
    public string Sk { get; set; } = string.Empty;

    [DynamoDbAttribute("userId")]
    public string UserId { get; set; } = string.Empty;

    [DynamoDbAttribute("name")]
    public string Name { get; set; } = string.Empty;

    // Sensitive fields - will be redacted in logs
    [Sensitive]
    [DynamoDbAttribute("email")]
    public string Email { get; set; } = string.Empty;

    [Sensitive]
    [DynamoDbAttribute("ssn")]
    public string SocialSecurityNumber { get; set; } = string.Empty;

    [Sensitive]
    [DynamoDbAttribute("creditCard")]
    public string CreditCardNumber { get; set; } = string.Empty;

    [DynamoDbAttribute("status")]
    public string Status { get; set; } = string.Empty;
}

Redaction Patterns​

Default Redaction​

By default, sensitive fields are replaced with [REDACTED]:

// When logging this operation:
await table.Users.PutAsync(new User
{
    UserId = "user123",
    Name = "John Doe",
    Email = "[email protected]",  // Will be logged as [REDACTED]
    SocialSecurityNumber = "123-45-6789"  // Will be logged as [REDACTED]
});

// Log output:
// PUT User: { UserId: "user123", Name: "John Doe", Email: "[REDACTED]", SocialSecurityNumber: "[REDACTED]" }

Custom Redaction​

Configure custom redaction patterns:

var options = new FluentDynamoDbOptions()
    .WithLogging(loggerFactory, new RedactionOptions
    {
        RedactionText = "***",
        PartialRedaction = true,
        PartialRedactionVisibleChars = 4
    });

With partial redaction enabled:

Email: "john***" (first 4 chars visible)
CreditCard: "****5678" (last 4 chars visible)

Usage Examples​

Query with Logging​

// Queries are logged with redacted filter values
var users = await table.Users.Query()
    .Where(x => x.Status == "active")
    .ToListAsync();

// Log output:
// QUERY Users WHERE Status = "active" - Returned 5 items

Update with Logging​

await table.Users.Update("user123")
    .Set(x => new { Email = "[email protected]" })
    .UpdateAsync();

// Log output:
// UPDATE User "user123" SET Email = "[REDACTED]"

Conditional Operations​

await table.Users.Put(user)
    .Where(x => x.Email.AttributeNotExists())
    .PutAsync();

// Log output:
// PUT User WHERE Email attribute_not_exists - Success

Get Operations​

var user = await table.Users.GetAsync("user123");

// Log output:
// GET User "user123" - Found
// Response: { UserId: "user123", Name: "John Doe", Email: "[REDACTED]" }

Delete Operations​

await table.Users.DeleteAsync("user123");

// Log output:
// DELETE User "user123" - Success

Compliance​

Automatic redaction helps meet compliance requirements:

GDPR​

  • Personal data is automatically redacted in logs
  • Supports data minimization principles
  • Audit trails don't expose PII

HIPAA​

  • Protected Health Information (PHI) can be marked sensitive
  • Logs remain compliant with disclosure rules

PCI-DSS​

  • Credit card numbers are redacted
  • Cardholder data protected in logs
  • Supports audit requirements without exposing sensitive data

Best Practices​

  1. Mark all PII as sensitive - Email, phone, SSN, addresses
  2. Mark financial data - Credit cards, bank accounts, salaries
  3. Mark authentication data - Passwords, tokens, API keys
  4. Use structured logging - Enables better log analysis while maintaining redaction
  5. Test redaction - Verify sensitive data doesn't appear in logs during development