Configuration

Configuration is an essential aspect of any cloud-native application. Effect simplifies the process of managing configuration by offering a convenient interface for configuration providers.

The configuration front-end in Effect enables ecosystem libraries and applications to specify their configuration requirements in a declarative manner. It offloads the complex tasks to a ConfigProvider, which can be supplied by third-party libraries.

Effect comes bundled with a straightforward default ConfigProvider that retrieves configuration data from environment variables. This default provider can be used during development or as a starting point before transitioning to more advanced configuration providers.

To make our application configurable, we need to understand three essential elements:

• Config Description: We describe the configuration data using an instance of Config<A>. If the configuration data is simple, such as a string, number, or boolean, we can use the built-in functions provided by the Config module. For more complex data types like HostPort, we can combine primitive configs to create a custom configuration description.

• Config Frontend: We utilize the instance of Config<A> to load the configuration data described by the instance (a Config is, in itself, an effect). This process leverages the current ConfigProvider to retrieve the configuration.

• Config Backend: The ConfigProvider serves as the underlying engine that manages the configuration loading process. Effect comes with a default config provider as part of its default services. This default provider reads the configuration data from environment variables. If we want to use a custom config provider, we can utilize the Layer.setConfigProvider layer to configure the Effect runtime accordingly.

Basic Configuration Types

Effect provides several built-in types for configuration values, which you can use right out of the box:

Type	Description
string	Constructs a config for a string value.
number	Constructs a config for a float value.
boolean	Constructs a config for a boolean value.
integer	Constructs a config for an integer value.
date	Constructs a config for a date value.
literal	Constructs a config for a literal (*) value.
logLevel	Constructs a config for a LogLevel value.
duration	Constructs a config for a duration value.
redacted	Constructs a config for a secret value.

(*) string | number | boolean | null | bigint

Example (Using Primitives)

Here’s an example of loading a basic configuration using environment variables for HOST and PORT:

primitives.ts

import { Effect, Config } from "effect"

// Define a program that loads HOST and PORT configuration
const program = Effect.gen(function* () {
  // Load the HOST from environment variables
  const host = yield* Config.string("HOST")
  // Load the PORT as a number
  const port = yield* Config.number("PORT")

  console.log(`Application started: ${host}:${port}`)
})

Effect.runPromise(program)

If you run this without setting the required environment variables:

npx tsx primitives.ts

you’ll see an error indicating the missing configuration:

[Error: (Missing data at HOST: "Expected HOST to exist in the process context")] {
  name: '(FiberFailure) Error',
  [Symbol(effect/Runtime/FiberFailure)]: Symbol(effect/Runtime/FiberFailure),
  [Symbol(effect/Runtime/FiberFailure/Cause)]: {
    _tag: 'Fail',
    error: {
      _op: 'MissingData',
      path: [ 'HOST' ],
      message: 'Expected HOST to exist in the process context'
    }
  }
}

To run the program successfully, set the environment variables as shown below:

HOST=localhost PORT=8080 npx tsx primitives.ts

Output:

Application started: localhost:8080

Default Values

Sometimes, you may encounter situations where an environment variable is missing, leading to an incomplete configuration. To address this, Effect provides the Config.withDefault function, which allows you to specify a default value. This fallback ensures that your application continues to function even if a required environment variable is not set.

Example (Using Default Values)

defaults.ts

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  const host = yield* Config.string("HOST")
  // Use default 8080 if PORT is not set
  const port = yield* Config.number("PORT").pipe(Config.withDefault(8080))
  console.log(`Application started: ${host}:${port}`)
})

Effect.runPromise(program)

Running this program with only the HOST environment variable set:

HOST=localhost npx tsx defaults.ts

produces the following output:

Application started: localhost:8080

In this case, even though the PORT environment variable is not set, the program continues to run, using the default value of 8080 for the port. This ensures that the application remains functional without requiring every configuration to be explicitly provided.

Constructors for Configurations

Effect provides several built-in constructors that allow you to define and manipulate configurations. These constructors take a Config as input and produce another Config, enabling more complex configuration structures.

Here are some of the key constructors:

Constructor	Description
array	Constructs a configuration for an array of values.
chunk	Constructs a configuration for a sequence of values.
option	Returns an optional configuration. If the data is missing, the result will be None; otherwise, it will be Some.
repeat	Describes a sequence of values, each following the structure of the given config.
hashSet	Constructs a configuration for a set of values.
hashMap	Constructs a configuration for a key-value map.

Additionally, there are three special constructors for specific use cases:

Constructor	Description
succeed	Constructs a config that contains a predefined value.
fail	Constructs a config that fails with the specified error message.
all	Combines multiple configurations into a tuple, struct, or argument list.

Example (Using array Constructor)

The following example demonstrates how to load an environment variable as an array of strings using the Config.array constructor.

array.ts

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  // Load array from env var MY_ARRAY
  const config = yield* Config.array(Config.string(), "MY_ARRAY")
  console.log(config)
})

Effect.runPromise(program)

If we run this program with the following environment variable:

MY_ARRAY=a,b,c npx tsx array.ts

The output will be:

[ 'a', 'b', 'c' ]

This shows how the array constructor converts a comma-separated string from an environment variable into an array of values, making configuration handling more flexible.

Operators

Effect provides several built-in operators to work with configurations, allowing you to manipulate and transform them according to your needs.

Transforming Operators

These operators enable you to modify configurations or validate their values:

Operator	Description
validate	Ensures that a configuration meets certain criteria, returning a validation error if it does not.
map	Transforms the values of a configuration using a provided function.
mapAttempt	Similar to map, but catches any errors thrown by the function and converts them into validation errors.
mapOrFail	Like map, but the function can fail. If it does, the result is a validation error.

Example (Using validate Operator)

validate.ts

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  // Load the NAME environment variable and validate its length
  const config = yield* Config.string("NAME").pipe(
    Config.validate({
      message: "Expected a string at least 4 characters long",
      validation: (s) => s.length >= 4
    })
  )
  console.log(config)
})

Effect.runPromise(program)

If we run this program with an invalid NAME value:

NAME=foo npx tsx validate.ts

The output will be:

[Error: (Invalid data at NAME: "Expected a string at least 4 characters long")] {
  name: '(FiberFailure) Error',
  [Symbol(effect/Runtime/FiberFailure)]: Symbol(effect/Runtime/FiberFailure),
  [Symbol(effect/Runtime/FiberFailure/Cause)]: {
    _tag: 'Fail',
    error: {
      _op: 'InvalidData',
      path: [ 'NAME' ],
      message: 'Expected a string at least 4 characters long'
    }
  }
}

Fallback Operators

Fallback operators are useful when you want to provide alternative configurations in case of errors or missing data. These operators ensure that your program can still run even if some configuration values are unavailable.

Operator	Description
orElse	Attempts to use the primary config first. If it fails or is missing, it falls back to another config.
orElseIf	Similar to orElse, but it switches to the fallback config only if the error matches a condition.

Example (Using orElse for Fallback)

In this example, the program requires two configuration values: A and B. We set up two configuration providers, each containing only one of the required values. Using the orElse operator, we combine these providers so the program can retrieve both A and B.

orElse.ts

import { Config, ConfigProvider, Effect, Layer } from "effect"

// A program that requires two configurations: A and B
const program = Effect.gen(function* () {
  const A = yield* Config.string("A") // Retrieve config A
  const B = yield* Config.string("B") // Retrieve config B
  console.log(`A: ${A}, B: ${B}`)
})

// First provider has A but is missing B
const provider1 = ConfigProvider.fromMap(new Map([["A", "A"]]))

// Second provider has B but is missing A
const provider2 = ConfigProvider.fromMap(new Map([["B", "B"]]))

// Use `orElse` to fall back from provider1 to provider2
const layer = Layer.setConfigProvider(
  provider1.pipe(ConfigProvider.orElse(() => provider2))
)

Effect.runPromise(Effect.provide(program, layer))

If we run this program:

npx tsx orElse.ts

The output will be:

A: A, B: B

Tip

In this example, we use ConfigProvider.fromMap to create a configuration provider from a simple JavaScript Map. This is particularly useful for testing, as described in the Testing Configurable Services section.

Custom Configurations

Effect allows you to define configurations for custom types by combining primitive configs using Config operators (such as zip, orElse, map) and constructors (like array, hashSet).

For example, let’s create a HostPort class, which has two fields: host and port.

class HostPort {
  constructor(readonly host: string, readonly port: number) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

To define a configuration for this custom type, we can combine primitive configs for string and number:

import { Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

// Combine the configuration for 'HOST' and 'PORT'
const both = Config.all([Config.string("HOST"), Config.number("PORT")])

// Map the configuration values into a HostPort instance
const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

In this example, Config.all(configs) combines two primitive configurations, Config<string> and Config<number>, into a Config<[string, number]>. The Config.map operator is then used to transform these values into an instance of the HostPort class.

Example (Using Custom Configuration)

App.ts

import { Effect, Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

// Combine the configuration for 'HOST' and 'PORT'
const both = Config.all([Config.string("HOST"), Config.number("PORT")])

// Map the configuration values into a HostPort instance
const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

// Main program that reads configuration and starts the application
const program = Effect.gen(function* () {
  const hostPort = yield* config
  console.log(`Application started: ${hostPort.url}`)
})

Effect.runPromise(program)

When you run this program, it will try to retrieve the values for HOST and PORT from your environment variables:

HOST=localhost PORT=8080 npx tsx App.ts

If successful, it will print:

Application started: localhost:8080

Top-level and Nested Configurations

We’ve seen how to define configurations at the top level, whether for primitive or custom types. In some cases, though, you might want to structure your configurations in a more nested way, organizing them under common namespaces for clarity and manageability.

For instance, consider the following ServiceConfig type:

class ServiceConfig {
  constructor(
    readonly host: string,
    readonly port: number,
    readonly timeout: number
  ) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

If you were to use this configuration in your application, it would expect the HOST, PORT, and TIMEOUT environment variables at the top level. But in many cases, you may want to organize configurations under a shared namespace—for example, grouping HOST and PORT under a SERVER namespace, while keeping TIMEOUT at the root.

To do this, you can use the Config.nested operator, which allows you to nest configuration values under a specific namespace. Let’s update the previous example to reflect this:

import { Config } from "effect"

class ServiceConfig {
  constructor(
    readonly host: string,
    readonly port: number,
    readonly timeout: number
  ) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

const serverConfig = Config.all([
  Config.string("HOST"),
  Config.number("PORT")
])

const serviceConfig = Config.map(
  Config.all([
    // Read 'HOST' and 'PORT' from 'SERVER' namespace
    Config.nested(serverConfig, "SERVER"),
    // Read 'TIMEOUT' from the root namespace
    Config.number("TIMEOUT")
  ]),
  ([[host, port], timeout]) => new ServiceConfig(host, port, timeout)
)

Now, if you run your application with this configuration setup, it will look for the following environment variables:

• SERVER_HOST for the host value
• SERVER_PORT for the port value
• TIMEOUT for the timeout value

This structured approach keeps your configuration more organized, especially when dealing with multiple services or complex applications.

Testing Configurable Services

When testing services, there are times when you need to provide specific configurations for your tests. To simulate this, it’s useful to mock the configuration backend that reads these values.

You can achieve this using the ConfigProvider.fromMap constructor. This method allows you to create a configuration provider from a Map<string, string>, where the map represents the configuration data. You can then use this mock provider in place of the default one by calling Layer.setConfigProvider. This function returns a Layer that can override the default configuration for your tests.

Example (Mocking a Config Provider for Testing)

import { Config, ConfigProvider, Layer, Effect } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}
  get url() {
    return `${this.host}:${this.port}`
  }
}

const config = Config.map(
  Config.all([Config.string("HOST"), Config.number("PORT")]),
  ([host, port]) => new HostPort(host, port)
)

const program = Effect.gen(function* () {
  const hostPort = yield* config
  console.log(`Application started: ${hostPort.url}`)
})

// Create a mock config provider using a map with test data
const mockConfigProvider = ConfigProvider.fromMap(
  new Map([
    ["HOST", "localhost"],
    ["PORT", "8080"]
  ])
)

// Override the default config provider with the mock provider
const layer = Layer.setConfigProvider(mockConfigProvider)

// Run the program using the mock config provider
Effect.runPromise(Effect.provide(program, layer))
// Output: Application started: localhost:8080

This approach helps you create isolated tests that don’t rely on external environment variables, ensuring your tests run consistently with mock configurations.

Redacted

The Config.redacted function is used to handle sensitive information safely. It parses the configuration value and wraps it in a Redacted<string>, a specialized data type designed to protect secrets.

When you log a Redacted value using console.log, the actual content remains hidden, providing an extra layer of security. To access the real value, you must explicitly use Redacted.value.

Example (Handling Redacted Values)

redacted.ts

import { Effect, Config, Redacted } from "effect"

const program = Effect.gen(function* () {
  const redacted = yield* Config.redacted("API_KEY")

  // Log the redacted value, which won't reveal the actual secret
  console.log(`Console output: ${redacted}`)

  // Access the real value using Redacted.value and log it
  console.log(`Actual value: ${Redacted.value(redacted)}`)
})

Effect.runPromise(program)

When this program is executed:

API_KEY=my-api-key tsx redacted.ts

The output will look like this:

Console output: <redacted>
Actual value: my-api-key

As shown, when logging the Redacted value using console.log, the output is <redacted>, ensuring that sensitive data remains concealed. However, by using Redacted.value, the true value ("my-api-key") can be accessed and displayed, providing controlled access to the secret.

Secret Deprecated

Deprecated since version 3.3.0: Please use Config.redacted for handling sensitive information going forward.

The Config.secret function was previously used to secure sensitive information in a similar way to Config.redacted. It wraps configuration values in a Secret type, which also conceals details when logged but allows access via Secret.value.

Example (Using Deprecated Config.secret)

secret.ts

import { Effect, Config, Secret } from "effect"

const program = Effect.gen(function* () {
  const secret = yield* Config.secret("API_KEY")

  // Log the secret value, which won't reveal the actual secret
  console.log(`Console output: ${secret}`)

  // Access the real value using Secret.value and log it
  console.log(`Actual value: ${Secret.value(secret)}`)
})

Effect.runPromise(program)

When this program is executed:

API_KEY=my-api-key tsx secret.ts

The output will look like this:

Console output: Secret(<redacted>)
Actual value: my-api-key