Configuration

Configuration is an essential aspect of any cloud-native application. Effect simplifies the process of managing configuration by offering a convenient interface for configuration providers.

The configuration front-end in Effect enables ecosystem libraries and applications to specify their configuration requirements in a declarative manner. It offloads the complex tasks to a ConfigProvider, which can be supplied by third-party libraries.

Effect comes bundled with a straightforward default ConfigProvider that retrieves configuration data from environment variables. This default provider can be used during development or as a starting point before transitioning to more advanced configuration providers.

To make our application configurable, we need to understand three essential elements:

• Config Description: We describe the configuration data using an instance of Config<A>. If the configuration data is simple, such as a string, number, or boolean, we can use the built-in functions provided by the Config module. For more complex data types like HostPort, we can combine primitive configs to create a custom configuration description.

• Config Frontend: We utilize the instance of Config<A> to load the configuration data described by the instance (a Config is, in itself, an effect). This process leverages the current ConfigProvider to retrieve the configuration.

• Config Backend: The ConfigProvider serves as the underlying engine that manages the configuration loading process. Effect comes with a default config provider as part of its default services. This default provider reads the configuration data from environment variables. If we want to use a custom config provider, we can utilize the Layer.setConfigProvider layer to configure the Effect runtime accordingly.

Getting Started

Effect provides a set of primitives for the most common types like string, number, boolean, integer, etc.

Let’s start with a simple example of how to read configuration from environment variables:

Using Effect.gen

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  const host = yield* Config.string("HOST")
  const port = yield* Config.number("PORT")
  console.log(`Application started: ${host}:${port}`)
})

Effect.runSync(program)

Using pipe

import { Effect, Config, Console } from "effect"

const program = Effect.all([
  Config.string("HOST"),
  Config.number("PORT")
]).pipe(
  Effect.andThen(([host, port]) =>
    Console.log(`Application started: ${host}:${port}`)
  )
)

Effect.runSync(program)

If we run this program we will get the following output:

npx tsx primitives.ts
(Missing data at HOST: "Expected HOST to exist in the process context")

This is because we have not provided any configuration. Let’s try running it with the following environment variables:

HOST=localhost PORT=8080 npx tsx primitives.ts
Application started: localhost:8080

Primitives

Effect offers these basic types out of the box:

Type	Description
string	Constructs a config for a string value.
number	Constructs a config for a float value.
boolean	Constructs a config for a boolean value.
integer	Constructs a config for an integer value.
date	Constructs a config for a date value.
literal	Constructs a config for a literal (*) value.
logLevel	Constructs a config for a LogLevel value.
duration	Constructs a config for a duration value.
redacted	Constructs a config for a secret value.

(*) string | number | boolean | null | bigint

Default Values

In some cases, you may encounter situations where an environment variable is not set, leading to a missing value in the configuration. To handle such scenarios, Effect provides the Config.withDefault function. This function allows you to specify a fallback or default value to use when an environment variable is not present.

Here’s how you can use Config.withDefault to handle fallback values:

Using Effect.gen

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  const host = yield* Config.string("HOST")
  const port = yield* Config.number("PORT").pipe(Config.withDefault(8080))
  console.log(`Application started: ${host}:${port}`)
})

Effect.runSync(program)

Using pipe

import { Effect, Config, Console } from "effect"

const program = Effect.all([
  Config.string("HOST"),
  Config.number("PORT").pipe(Config.withDefault(8080))
]).pipe(
  Effect.andThen(([host, port]) =>
    Console.log(`Application started: ${host}:${port}`)
  )
)

Effect.runSync(program)

When running the program with the command:

HOST=localhost npx tsx withDefault.ts

you will see the following output:

Application started: localhost:8080

Even though the PORT environment variable is not set, the fallback value of 8080 is used, ensuring that the program continues to run smoothly with a default value.

Constructors

Effect provides several built-in constructors. These are functions that take a Config as input and produce another Config.

• array: Constructs a config for an array of values.
• chunk: Constructs a config for a sequence of values.
• option: Returns an optional version of this config, which will be None if the data is missing from configuration, and Some otherwise.
• repeat: Returns a config that describes a sequence of values, each of which has the structure of this config.
• hashSet: Constructs a config for a sequence of values.
• hashMap: Constructs a config for a sequence of values.

In addition to the basic ones, there are three special constructors you might find useful:

• succeed: Constructs a config which contains the specified value.
• fail: Constructs a config that fails with the specified message.
• all: Constructs a config from a tuple / struct / arguments of configs.

Example

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  const config = yield* Config.array(Config.string(), "MY_ARRAY")
  console.log(config)
})

Effect.runSync(program)

MY_ARRAY=a,b,c npx tsx array.ts

[ 'a', 'b', 'c' ]

Operators

Effect comes with a set of built-in operators to help you manipulate and handle configurations.

Transforming Operators

These operators allow you to transform a config into a new one:

• validate: Returns a config that describes the same structure as this one, but which performs validation during loading.
• map: Creates a new config with the same structure as the original but with values transformed using a given function.
• mapAttempt: Similar to map, but if the function throws an error, it’s caught and turned into a validation error.
• mapOrFail: Like map, but allows for functions that might fail. If the function fails, it results in a validation error.

Example

import { Effect, Config } from "effect"

const program = Effect.gen(function* () {
  const config = yield* Config.string("NAME").pipe(
    Config.validate({
      message: "Expected a string at least 4 characters long",
      validation: (s) => s.length >= 4
    })
  )
  console.log(config)
})

Effect.runSync(program)

NAME=foo npx tsx validate.ts

[(Invalid data at NAME: "Expected a string at least 4 characters long")]

Fallback Operators

These operators help you set up fallbacks in case of errors or missing data:

• orElse: Sets up a config that tries to use this config first. If there’s an issue, it falls back to another specified config.
• orElseIf: This one also tries to use the main config first but switches to a fallback config if there’s an error that matches a specific condition.

Example

In this example, we have a program that requires two configurations: A and B. We will use two configuration providers, where each provider has only one of the configurations. We will demonstrate how to set up fallbacks using the orElse operator.

import { Config, ConfigProvider, Effect, Layer } from "effect"

// A program that requires two configurations: A and B
const program = Effect.gen(function* () {
  const A = yield* Config.string("A")
  const B = yield* Config.string("B")
  console.log(`A: ${A}`, `B: ${B}`)
})

const provider1 = ConfigProvider.fromMap(
  new Map([
    ["A", "A"]
    // B is missing
  ])
)

const provider2 = ConfigProvider.fromMap(
  new Map([
    // A is missing
    ["B", "B"]
  ])
)

const layer = Layer.setConfigProvider(
  provider1.pipe(ConfigProvider.orElse(() => provider2))
)

Effect.runSync(Effect.provide(program, layer))

npx tsx orElse.ts

A: A B: B

Note

The ConfigProvider.fromMap function used in this example creates a configuration provider from a Map. This behavior is explained in more detail in the Testing Services section.

Custom Configurations

In addition to primitive types, we can also define configurations for custom types. To achieve this, we use primitive configs and combine them using Config operators (zip, orElse, map, etc.) and constructors (array, hashSet, etc.).

Let’s consider the HostPort data type, which consists of two fields: host and port.

class HostPort {
  constructor(readonly host: string, readonly port: number) {}
}

We can define a configuration for this data type by combining primitive configs for string and number:

import { Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

const both = Config.all([Config.string("HOST"), Config.number("PORT")])

const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

In the above example, we use the Config.all(configs) operator to combine two primitive configs Config<string> and Config<number> into a Config<[string, number]>.

If we use this customized configuration in our application:

Using Effect.gen

import { Effect, Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

const both = Config.all([Config.string("HOST"), Config.number("PORT")])

const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

const program = Effect.gen(function* () {
  const hostPort = yield* config
  console.log(`Application started: ${hostPort.url}`)
})

Using pipe

import { Effect, Config, Console } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

const both = Config.all([Config.string("HOST"), Config.number("PORT")])

const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

const program = config.pipe(
  Effect.andThen((hostPort) =>
    Console.log(`Application started: ${hostPort.url}`)
  )
)

when you run the program using Effect.runSync(program), it will attempt to read the corresponding values from environment variables (HOST and PORT):

HOST=localhost PORT=8080 npx tsx App.ts

Application started: localhost:8080

Top-level and Nested Configurations

So far, we have learned how to define configurations in a top-level manner, whether they are for primitive or custom types. However, we can also define nested configurations.

Let’s assume we have a ServiceConfig data type that consists of two fields: hostPort and timeout.

import { Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

class ServiceConfig {
  constructor(readonly hostPort: HostPort, readonly timeout: number) {}
}

If we use this customized config in our application, it tries to read corresponding values from environment variables: HOST, PORT, and TIMEOUT.

However, in many cases, we don’t want to read all configurations from the top-level namespace. Instead, we may want to nest them under a common namespace. For example, we want to read both HOST and PORT from the HOSTPORT namespace, and TIMEOUT from the root namespace.

To achieve this, we can use the Config.nested combinator. It allows us to nest configs under a specific namespace. Here’s how we can update our configuration:

import { Config } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

const both = Config.all([Config.string("HOST"), Config.number("PORT")])

const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

class ServiceConfig {
  constructor(readonly hostPort: HostPort, readonly timeout: number) {}
}

const serfviceConfig = Config.map(
  Config.all([
    Config.nested(config, "HOSTPORT"),
    Config.number("TIMEOUT")
  ]),
  ([hostPort, timeout]) => new ServiceConfig(hostPort, timeout)
)

Now, if we run our application, it will attempt to read the corresponding values from the environment variables: HOSTPORT_HOST, HOSTPORT_PORT, and TIMEOUT.

Testing Services

When testing services, there are scenarios where we need to provide specific configurations to them. In such cases, we should be able to mock the backend that reads the configuration data.

To accomplish this, we can use the ConfigProvider.fromMap constructor. This constructor takes a Map<string, string> that represents the configuration data, and it returns a config provider that reads the configuration from that map.

Once we have the mock config provider, we can use Layer.setConfigProvider function. This function allows us to override the default config provider and provide our own custom config provider. It returns a Layer that can be used to configure the Effect runtime for our test specs.

Here’s an example of how we can mock a config provider for testing purposes:

import { Config, ConfigProvider, Layer, Effect } from "effect"

class HostPort {
  constructor(readonly host: string, readonly port: number) {}

  get url() {
    return `${this.host}:${this.port}`
  }
}

const both = Config.all([Config.string("HOST"), Config.number("PORT")])

const config = Config.map(
  both,
  ([host, port]) => new HostPort(host, port)
)

const program = Effect.gen(function* () {
  const hostPort = yield* config
  console.log(`Application started: ${hostPort.url}`)
})

// Create a mock config provider using ConfigProvider.fromMap
const mockConfigProvider = ConfigProvider.fromMap(
  new Map([
    ["HOST", "localhost"],
    ["PORT", "8080"]
  ])
)

// Create a layer using Layer.setConfigProvider to override the default config provider
const layer = Layer.setConfigProvider(mockConfigProvider)

// Run the program using the provided layer
Effect.runSync(Effect.provide(program, layer))
// Output: Application started: localhost:8080

By using this approach, we can easily mock the configuration data and test our services with different configurations in a controlled manner.

Redacted

What sets Config.redacted apart from Config.string is its handling of sensitive information. It parses the config value and wraps it in a Redacted<string>, a data type designed for holding secrets.

When you use Console.log on a redacted, the actual value remains hidden, providing an added layer of security. The only way to access the value is by using Redacted.value.

import { Effect, Config, Console, Redacted } from "effect"

const program = Config.redacted("API_KEY").pipe(
  Effect.tap((redacted) => Console.log(`Console output: ${redacted}`)),
  Effect.tap((redacted) =>
    Console.log(`Actual value: ${Redacted.value(redacted)}`)
  )
)

Effect.runSync(program)

If we run this program we will get the following output:

API_KEY=my-api-key tsx Redacted.ts
Console output: <redacted>
Actual value: my-api-key

In this example, you can see that when logging the redacted using Console.log, the actual value is replaced with <redacted>, ensuring that sensitive information is not exposed. The Redacted.value function, on the other hand, provides a controlled way to retrieve the original secret value.

Secret Deprecated

Deprecated since version 3.3.0: Please use Config.redacted for handling sensitive information going forward.

Config.secret functions similarly to Config.redacted by securing sensitive information. It wraps configuration values in a Secret type, which also obscures details when logged but allows access through the Secret.value method.

import { Effect, Config, Console, Secret } from "effect"

const program = Config.secret("API_KEY").pipe(
  Effect.tap((secret) => Console.log(`Console output: ${secret}`)),
  Effect.tap((secret) =>
    Console.log(`Secret value: ${Secret.value(secret)}`)
  )
)

Effect.runSync(program)

If we run this program we will get the following output:

API_KEY=my-api-key tsx Secret.ts
Console output: Secret(<redacted>)
Secret value: my-api-key