Introduction

nyl (pronounced like “nil”) is a Kubernetes manifest generator with Helm integration, designed to simplify the management of Kubernetes resources through a powerful templating system.

Goals

nyl aims to provide:

• High performance manifest generation
• Lightweight binary for easy distribution
• Clean, maintainable architecture for future development
• Powerful templating with Helm integration

Features

• Configuration loading (YAML, JSON)
• File discovery with upward directory traversal
• nyl validate command with strict mode
• nyl new project command for project scaffolding
• nyl new component command for component scaffolding
• Helm integration and component discovery
• Template rendering with Jinja2
• Kubernetes operations (diff, apply)
• Release state management with revision tracking

Key Features

• Safety: Memory safety without garbage collection
• Performance: Compiled binaries with minimal runtime overhead
• Concurrency: Efficient concurrent operations
• Reliability: Strong type system catches errors at compile time
• Rich ecosystem: Extensive library support

Architecture

nyl is structured into several key modules:

• config: Project configuration loading and validation
• cli: Command-line interface and argument parsing
• template: Jinja2 template rendering (Phase 3)
• kubernetes: Kubernetes client integration (Phase 4)
• resources: Resource definitions and transformations
• generator: Manifest generation pipeline (Phase 3+)

Getting Started

This guide will help you get started with nyl-rs.

Installation

From Source

cd nyl-rs
cargo build --release

The binary will be available at target/release/nyl.

Install Locally

cargo install --path .

This installs nyl to ~/.cargo/bin/nyl.

Quick Start

1. Create a New Project

nyl new project my-app
cd my-app

This creates:

• nyl-project.yaml - Project configuration
• components/ - Directory for components

2. Add a Component

nyl new component v1.example.io MyApp

This creates a new component at components/v1.example.io/MyApp/ with:

• Chart.yaml - Helm chart metadata
• values.yaml - Default values
• values.schema.json - JSON schema for validation
• templates/deployment.yaml - Kubernetes deployment template

3. Validate Your Project

nyl validate

Output:

✓ Found project config: /path/to/my-app/nyl-project.yaml
✓ Components directory exists: /path/to/my-app/components
✓ Search path exists: /path/to/my-app

✓ Validation passed

4. Strict Validation

For CI/CD pipelines, use strict mode to treat warnings as errors:

nyl validate --strict

Project Structure

my-app/
├── nyl-project.yaml          # Project configuration
├── components/               # Component definitions
│   └── v1.example.io/
│       └── MyApp/
│           ├── Chart.yaml
│           ├── values.yaml
│           ├── values.schema.json
│           └── templates/
│               └── deployment.yaml
└── lib/                      # Optional: shared libraries (Phase 2+)

Next Steps

• Read about Configuration
• Learn about the new command
• Learn about the validate command
• Check the Migration Guide if you’re coming from Python nyl

Configuration

nyl uses a project configuration file to define project settings. The configuration file can be in YAML or JSON format.

Configuration File

nyl searches for configuration files in the following order:

1. nyl-project.yaml
2. nyl-project.json

The search starts in the current directory and moves upward through parent directories until a configuration file is found.

Configuration Structure

YAML Format

settings:
  generate_applysets: false
  on_lookup_failure: Error
  components_path: components
  search_path:
    - .
    - lib

JSON Format

{
  "settings": {
    "generate_applysets": false,
    "on_lookup_failure": "Error",
    "components_path": "components",
    "search_path": [
      ".",
      "lib"
    ]
  }
}

Settings

generate_applysets

• Type: boolean
• Default: false
• Description: If enabled, automatically generate an ApplySet for every template file. (Phase 3+)

on_lookup_failure

• Type: string
• Default: "Error"
• Valid Values: "Error", "CreatePlaceholder", "SkipResource"
• Description: Behavior when a lookup() call fails during template rendering. (Phase 3+)
  • Error: Fail with an error
  • CreatePlaceholder: Create a placeholder resource
  • SkipResource: Skip the resource that depends on the lookup

components_path

• Type: string (path)
• Default: "components"
• Description: Path to the directory that contains nyl components. Relative paths are resolved relative to the configuration file location.

search_path

• Type: array of string (paths)
• Default: ["."]
• Description: Search paths for additional resources used by the project. Used for example when using the chart.path option on a HelmChart resource. (Phase 2+) Relative paths are resolved relative to the configuration file location.

Path Resolution

All relative paths in the configuration are resolved relative to the configuration file’s parent directory.

For example, if your configuration is at /home/user/my-app/nyl-project.yaml:

settings:
  components_path: components        # Resolves to /home/user/my-app/components
  search_path:
    - .                              # Resolves to /home/user/my-app
    - lib                            # Resolves to /home/user/my-app/lib
    - /absolute/path                 # Remains /absolute/path

Default Configuration

If no configuration file is found, nyl uses the following defaults:

settings:
  generate_applysets: false
  on_lookup_failure: Error
  components_path: null              # Defaults to ./components
  search_path:
    - .

Validation

Use nyl validate to check your configuration:

nyl validate

This checks:

• Configuration file syntax (YAML/JSON)
• on_lookup_failure has a valid value
• Components directory exists
• Search paths are accessible

Use --strict mode in CI/CD to treat warnings as errors:

nyl validate --strict

Future Settings (Phase 2+)

The configuration file may also contain:

• profiles: Profile configurations for different environments
• secrets: Secret provider configurations

These sections are ignored in Phase 1 but will be supported in future phases.

Git Integration

Nyl provides built-in Git repository management for fetching Helm charts and scanning application manifests from Git repositories. This enables declarative infrastructure management without requiring manual repository cloning.

Features

• Bare repositories: Minimal disk usage with shared object store
• Worktrees: Isolated checkouts for different refs (branches, tags, commits)
• Lazy fetching: Refs fetched first, objects downloaded on-demand
• Automatic caching: Repositories cached locally for fast subsequent access
• Concurrent access: Multiple refs from the same repository can be checked out simultaneously

Cache Directory

Nyl stores Git repositories in a cache directory to avoid redundant clones and improve performance.

Configuration

The cache directory is determined by:

1. NYL_CACHE_DIR environment variable (preferred)
2. .nyl/cache/ in the current directory (fallback)

Example:

export NYL_CACHE_DIR=/var/cache/nyl
nyl render app.yaml

Cache Structure

$NYL_CACHE_DIR/git/
├── bare/
│   ├── {url_hash}-{repo_name}/          # Bare repository
│   │   ├── objects/
│   │   ├── refs/
│   │   └── ...
│   └── {url_hash2}-{repo_name2}/
└── worktrees/
    ├── {url_hash}-{ref_hash}/            # Worktree checkout
    ├── {url_hash}-{ref_hash2}/           # Another ref from same repo
    └── {url_hash2}-{ref_hash}/           # Different repo

Key points:

• One bare repository per Git URL (shared object store)
• One worktree per unique URL + ref combination
• Worktrees share objects from bare repo (disk-efficient)
• URL and ref hashes ensure uniqueness and avoid conflicts

Using Git with HelmChart

HelmChart resources can reference Helm charts stored in Git repositories.

Basic Example

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: nginx
spec:
  chart:
    git: https://github.com/bitnami/charts.git
    git_ref: main
    path: bitnami/nginx
  release:
    name: nginx
    namespace: default

Parameters

• git: Git repository URL (required)
• git_ref: Branch, tag, or commit (optional, defaults to HEAD)
• path: Subdirectory within repository (optional)

Supported Ref Types

You can reference different types of Git refs:

Branch:

spec:
  chart:
    git: https://github.com/example/charts.git
    git_ref: main

Tag:

spec:
  chart:
    git: https://github.com/example/charts.git
    git_ref: v2.1.0

Commit SHA:

spec:
  chart:
    git: https://github.com/example/charts.git
    git_ref: abc123def456

HEAD (default):

spec:
  chart:
    git: https://github.com/example/charts.git
    # git_ref defaults to HEAD

Using Git with ApplicationGenerator

ApplicationGenerator resources scan Git repositories for Nyl manifests and automatically generate ArgoCD Applications.

Example

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: cluster-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/example/gitops-demo.git
    targetRevision: main
    path: apps
    include:
      - "*.yaml"
      - "*.yml"
    exclude:
      - ".*"
      - "_*"
  project: default

When rendered, Nyl will:

1. Clone the Git repository to cache
2. Check out the main branch
3. Navigate to the apps/ directory
4. Scan for YAML files matching the include/exclude patterns
5. Generate ArgoCD Application manifests for each NylRelease found

Performance Characteristics

Initial Clone

• Lazy fetching: Only refs are fetched initially (~KB), not objects (~MB/GB)
• On-demand objects: Commit objects fetched only when needed
• Bandwidth efficient: Minimal initial download

Subsequent Access

• Cache hit: Near-instant if ref already checked out
• Ref update: Only fetch new refs if branch updated
• Object reuse: Worktrees share objects from bare repo

Disk Usage

• Bare repo: One copy of objects for all refs
• Worktrees: Only working directory files (no .git directory)
• Efficient: Much smaller than full clones for each ref

Example disk usage for a 100MB repository with 3 refs:

• Traditional approach: 3 × 100MB = 300MB
• Nyl approach: 100MB (bare) + 3 × ~10MB (worktrees) = ~130MB

Authentication

Nyl supports both public and private Git repositories through multiple authentication methods.

Automatic Credential Discovery

When running inside a Kubernetes cluster with access to ArgoCD secrets, Nyl automatically discovers repository credentials from ArgoCD repository secrets. No additional configuration required!

Example: Using a private Helm chart

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: private-app
spec:
  chart:
    git: git@github.com:myorg/private-charts.git
    git_ref: main
    path: charts/app
  release:
    name: private-app
    namespace: default

If an ArgoCD repository secret exists for github.com, Nyl will automatically use those credentials.

Authentication Methods

1. ArgoCD Repository Secrets (Recommended)

   • Credentials automatically discovered from argocd namespace
   • Supports both SSH keys and HTTPS tokens
   • Zero configuration required
   • See Repository Secrets for details

2. SSH Agent

   • Fallback for SSH URLs when no secret found
   • Works with local development workflows
   • Requires SSH agent running with key loaded

3. Public Repositories

   • No authentication needed
   • Works out of the box

Supported Credential Types

SSH Key Authentication:

• Private key stored in ArgoCD secret
• Recommended for production use
• Better security than HTTPS tokens

HTTPS Token Authentication:

• Personal access tokens or passwords
• Useful for HTTPS-only repositories
• Stored in ArgoCD secret

Example: Creating Repository Secret

# SSH authentication
kubectl create secret generic github-private \
  -n argocd \
  --from-literal=url=git@github.com:myorg/charts.git \
  --from-file=sshPrivateKey=$HOME/.ssh/id_rsa

kubectl label secret github-private \
  -n argocd \
  argocd.argoproj.io/secret-type=repository

# HTTPS authentication
kubectl create secret generic github-https \
  -n argocd \
  --from-literal=url=https://github.com/myorg/charts.git \
  --from-literal=username=myuser \
  --from-literal=password=ghp_token123

kubectl label secret github-https \
  -n argocd \
  argocd.argoproj.io/secret-type=repository

For complete documentation on authentication, see Repository Secrets.

Limitations

1. Shallow clones not supported: libgit2 (the underlying library) doesn’t support shallow clones. Full repository history is fetched.

2. Force checkout: When reusing worktrees, local changes are discarded. Worktrees are treated as read-only checkouts.

Troubleshooting

Cache directory permissions

Problem: Permission denied when creating cache directory

Solution: Set NYL_CACHE_DIR to a writable location:

export NYL_CACHE_DIR=$HOME/.cache/nyl

Large repository performance

Problem: Initial clone is slow for large repositories

Explanation: Nyl fetches the full repository (no shallow clone support)

Workaround: Use a specific tag/commit to avoid fetching all branches:

spec:
  chart:
    git: https://github.com/large/repo.git
    git_ref: v1.2.3  # Specific tag, not a branch

Stale cache

Problem: Git repository not updating with latest changes

Solution: Clear the cache directory:

rm -rf $NYL_CACHE_DIR/git/

Or for a specific repository:

# Find the cached repo
ls $NYL_CACHE_DIR/git/bare/
# Remove it
rm -rf $NYL_CACHE_DIR/git/bare/{hash}-{repo-name}
rm -rf $NYL_CACHE_DIR/git/worktrees/{hash}-*

Authentication failures

Problem: Cannot access private repository

Solution: See Repository Secrets for authentication setup

Examples

Multi-environment Chart from Git

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: app-production
spec:
  chart:
    git: https://github.com/company/charts.git
    git_ref: stable
    path: applications/myapp
  release:
    name: myapp
    namespace: production
  values:
    environment: production
    replicas: 5

Development Branch

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: app-development
spec:
  chart:
    git: https://github.com/company/charts.git
    git_ref: develop
    path: applications/myapp
  release:
    name: myapp
    namespace: development
  values:
    environment: development
    replicas: 1

Specific Version

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: app-stable
spec:
  chart:
    git: https://github.com/company/charts.git
    git_ref: v2.1.0
    path: applications/myapp
  release:
    name: myapp
    namespace: staging

ApplicationGenerator with Filtering

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: monitoring-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/company/infrastructure.git
    targetRevision: main
    path: monitoring
    include:
      - "*.yaml"
      - "*.yml"
    exclude:
      - ".*"              # Hidden files
      - "_*"              # Files starting with underscore
      - "test_*"          # Test files
  project: monitoring
  labels:
    team: platform
    category: monitoring

Best Practices

1. Pin versions in production: Use tags or commit SHAs for production deployments:

   git_ref: v1.2.3  # Tag
   # or
   git_ref: abc123  # Commit SHA
   

2. Use branches for development: Use branch refs for development environments:

   git_ref: develop  # Branch name
   

3. Set cache directory: Configure NYL_CACHE_DIR in CI/CD environments:

   export NYL_CACHE_DIR=/cache/nyl
   

4. Monitor cache size: Periodically clean up old worktrees if disk space is limited:

   find $NYL_CACHE_DIR/git/worktrees -mtime +30 -delete
   

5. Use subpaths: Keep charts in subdirectories for better organization:

   spec:
     chart:
       git: https://github.com/company/charts.git
       path: charts/applications/myapp
   

Commands

nyl provides several commands for managing Kubernetes manifests:

Available Commands

Phase 1 (Current)

• new - Create new projects and components
• validate - Validate project configuration

Phase 2+ (Coming Soon)

• render - Render Kubernetes manifests
• diff - Show diff between rendered manifests and cluster state
• apply - Apply rendered manifests to the cluster

Global Options

--verbose / -v

Enable verbose logging for debugging.

nyl --verbose validate
nyl -v new project my-app

--help / -h

Show help information for any command.

nyl --help
nyl new --help
nyl validate --help

--version / -V

Show the version of nyl.

nyl --version

new

Create new nyl projects and components.

Synopsis

nyl new project <name> [--path <path>]
nyl new component <api-version> <kind>

# Legacy syntax (deprecated)
nyl new <name> [--path <path>]

Subcommands

project

Create a new nyl project with default structure.

Arguments:

• <name> - Name of the project

Options:

• --path <path> - Path where to create the project (default: current directory)

Example:

nyl new project my-app
cd my-app

Creates:

my-app/
├── nyl-project.yaml
└── components/

Default Configuration:

The created nyl-project.yaml contains:

settings:
  generate_applysets: false
  on_lookup_failure: Error
  components_path: components
  search_path:
    - .

component

Create a new component in the current project.

Arguments:

• <api-version> - Component API version (e.g., v1.example.io)
• <kind> - Component kind (e.g., MyApp)

Example:

nyl new component v1.example.io MyApp

Creates:

components/v1.example.io/MyApp/
├── Chart.yaml
├── values.yaml
├── values.schema.json
└── templates/
    └── deployment.yaml

Files Created:

• Chart.yaml - Helm chart metadata
• values.yaml - Default values for the component
• values.schema.json - JSON schema for value validation
• templates/deployment.yaml - Basic Kubernetes deployment template

Legacy Syntax

The legacy syntax nyl new <name> is deprecated but still supported for backward compatibility:

nyl new my-app

This will show a deprecation warning and create a project. Use nyl new project <name> instead.

Examples

Create a project in a specific directory

nyl new project my-app --path ~/projects
cd ~/projects/my-app

Create a complete project with component

# Create project
nyl new project my-app
cd my-app

# Create component
nyl new component v1.example.io WebServer

# Validate
nyl validate

Component Template Structure

The generated component follows Helm chart conventions:

Chart.yaml

apiVersion: v2
name: myapp
description: A Helm chart for MyApp
type: application
version: 0.1.0
appVersion: "1.0"

values.yaml

replicaCount: 1

image:
  repository: nginx
  pullPolicy: IfNotPresent
  tag: "latest"

service:
  type: ClusterIP
  port: 80

resources:
  limits:
    cpu: 100m
    memory: 128Mi
  requests:
    cpu: 100m
    memory: 128Mi

templates/deployment.yaml

A basic Kubernetes Deployment template using Helm templating syntax.

Error Handling

Project already exists

$ nyl new project existing-app
Error: Project directory already exists: ./existing-app

Component already exists

$ nyl new component v1.example.io MyApp
Error: Component already exists: ./components/v1.example.io/MyApp

No project configuration found

If you try to create a component without a project configuration:

$ nyl new component v1.example.io MyApp
# Creates component in ./components/ (default)

validate

Validate project configuration and check for common issues.

Synopsis

nyl validate [path] [--strict]

Arguments

• [path] - Path to the project directory (default: current directory)

Options

• --strict / -s - Treat warnings as errors (useful for CI/CD)

Description

The validate command checks your nyl project configuration for:

1. Configuration file existence - Warns if no config file is found
2. Configuration syntax - Validates YAML/JSON syntax
3. Components directory - Checks if components directory exists
4. Search paths - Verifies all search paths are accessible
5. Settings validation - Ensures on_lookup_failure has a valid value

Output

Success

$ nyl validate
✓ Found project config: /path/to/nyl-project.yaml
✓ Components directory exists: /path/to/components
✓ Search path exists: /path/to/lib

✓ Validation passed

With Warnings

$ nyl validate
✓ Found project config: /path/to/nyl-project.yaml
⚠ Components directory does not exist: /path/to/components
✓ Search path exists: /path/to/lib

Validation warnings:
  ⚠ Components directory does not exist: /path/to/components

✓ Validation passed with warnings

No Configuration

$ nyl validate
⚠ No project configuration file found, using defaults
⚠ Components directory does not exist: ./components
✓ Search path exists: .

Validation warnings:
  ⚠ Components directory does not exist: ./components

✓ Validation passed with warnings

Strict Mode

In strict mode, warnings are treated as errors. This is useful for CI/CD pipelines:

$ nyl validate --strict
✓ Found project config: /path/to/nyl-project.yaml
⚠ Components directory does not exist: /path/to/components
✓ Search path exists: /path/to/lib

Validation warnings:
  ⚠ Components directory does not exist: /path/to/components

✗ Validation failed in strict mode (warnings treated as errors)
Error: Validation failed in strict mode

Exit code: 1

Validation Checks

1. Configuration File

• Check: Configuration file exists
• Warning: No configuration file found (uses defaults)
• Error: Configuration file has syntax errors

2. Components Directory

• Check: Directory specified in components_path exists
• Warning: Directory does not exist
• Note: Creates warning even with default configuration

3. Search Paths

• Check: All paths in search_path exist
• Warning: Path does not exist for each missing path

4. on_lookup_failure Value

• Check: Value is one of: "Error", "CreatePlaceholder", "SkipResource"
• Warning: Invalid value specified

Examples

Validate current project

nyl validate

Validate specific project

nyl validate /path/to/project

Validate in CI/CD

# .github/workflows/validate.yml
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Validate nyl project
        run: nyl validate --strict

Validate with verbose logging

nyl --verbose validate

Output includes debug information:

INFO Validating project configuration
DEBUG Validation path: .
DEBUG Strict mode: false
✓ Found project config: /path/to/nyl-project.yaml
...

Error Handling

Invalid YAML

$ nyl validate
Error: Failed to parse YAML config: invalid YAML syntax

Invalid JSON

$ nyl validate
Error: Failed to parse JSON config: expected ',' or '}' at line 5

Invalid on_lookup_failure

$ nyl validate
✓ Found project config: /path/to/nyl-project.yaml

Validation warnings:
  ⚠ Invalid on_lookup_failure value 'InvalidValue'. Must be one of: Error, CreatePlaceholder, SkipResource

✓ Validation passed with warnings

Exit Codes

• 0 - Validation passed (with or without warnings in normal mode)
• 1 - Validation failed (syntax errors or warnings in strict mode)

render

Status: Phase 3 (Not yet implemented)

Render Kubernetes manifests from nyl components and templates.

Synopsis

nyl render [options]

Description

The render command will generate Kubernetes manifests by:

1. Loading project configuration
2. Discovering components
3. Rendering templates with Jinja2
4. Processing Helm charts
5. Outputting YAML manifests to stdout

This command will be implemented in Phase 3.

Planned Features

• Template rendering with Jinja2
• Helm chart integration
• Component discovery and loading
• Profile support for different environments
• Secret provider integration
• Applyset generation

Coming Soon

This command is planned for Phase 3 of the Rust rewrite.

diff

Status: Phase 4 (Not yet implemented)

Show the difference between rendered manifests and the current cluster state.

Synopsis

nyl diff [options]

Description

The diff command will:

1. Render manifests (like nyl render)
2. Connect to the Kubernetes cluster
3. Fetch current resource states
4. Show differences using a diff format

This command will be implemented in Phase 4.

Planned Features

• Kubernetes cluster connection
• Resource state fetching
• Smart diff formatting
• Dry-run validation
• Change summary

Coming Soon

This command is planned for Phase 4 of the Rust rewrite.

apply

Status: Phase 4 (Not yet implemented)

Apply rendered manifests to the Kubernetes cluster.

Synopsis

nyl apply [options]

Description

The apply command will:

1. Render manifests (like nyl render)
2. Connect to the Kubernetes cluster
3. Apply changes using kubectl apply semantics
4. Report on changes made

This command will be implemented in Phase 4.

Planned Features

• Kubernetes cluster connection
• Safe apply with server-side apply
• Dry-run mode
• Progress reporting
• Rollback support

Coming Soon

This command is planned for Phase 4 of the Rust rewrite.

generate

Generate auxiliary resources and configurations from Nyl manifests.

Usage

nyl generate <subcommand> [options]

Subcommands

argocd

Generate ArgoCD Application manifests from Nyl releases.

nyl generate argocd [OPTIONS] <PATH>

Arguments:

• <PATH>: Directory to scan for Nyl release files

Options:

• -o, --output <FILE>: Output file (default: stdout)
• --repo-url <URL>: Git repository URL for generated Applications
• --target-revision <REV>: Target revision (branch/tag/commit)
• --destination-server <URL>: Kubernetes server URL (default: https://kubernetes.default.svc)
• --destination-namespace <NS>: Namespace for Applications (default: argocd)
• --project <PROJECT>: ArgoCD project name (default: default)

Example:

# Generate Applications for all releases in clusters/default
nyl generate argocd clusters/default \
  --repo-url https://github.com/myorg/gitops.git \
  --target-revision main \
  -o applications.yaml

Relation to ApplicationGenerator

The nyl generate argocd command is a manual CLI tool for one-time generation of ArgoCD Applications from a directory of Nyl releases. It’s useful for:

• Initial migration to ArgoCD
• One-off Application generation
• CI/CD pipelines that don’t use ApplicationGenerator

The ApplicationGenerator resource is the recommended approach for ongoing management. It provides:

• Automatic discovery and generation during nyl render
• Integration with ArgoCD plugin for GitOps
• Declarative configuration
• Self-hosting bootstrap pattern

When to use each:

Recommendation: Use ApplicationGenerator for most use cases. It’s processed during nyl render and integrates seamlessly with ArgoCD.

See Also

• ApplicationGenerator Resource
• ArgoCD Bootstrapping
• render Command

ArgoCD Integration

Nyl provides first-class support for ArgoCD through a custom configuration management plugin and the ApplicationGenerator resource. This integration enables GitOps-style deployments where ArgoCD can render Nyl manifests directly from Git repositories.

Key Features

• ArgoCD Plugin: Process Nyl manifests directly within ArgoCD
• ApplicationGenerator: Automatically generate ArgoCD Applications from Nyl releases
• Self-Hosting Pattern: Bootstrap ArgoCD with Nyl using a declarative approach
• Live Diffing: See differences between Git state and cluster state
• Automated Sync: Optionally enable automatic synchronization and pruning

How It Works

The Plugin

Nyl provides an ArgoCD configuration management plugin that acts as a bridge between ArgoCD and Nyl. When ArgoCD syncs an Application that uses the Nyl plugin:

1. ArgoCD clones the Git repository
2. The Nyl plugin is invoked to render manifests
3. Nyl processes the YAML files (HelmCharts, NylRelease, etc.)
4. Rendered Kubernetes manifests are returned to ArgoCD
5. ArgoCD applies the manifests to the cluster

ApplicationGenerator

The ApplicationGenerator resource enables automatic discovery and generation of ArgoCD Applications. When you use nyl render on a file containing an ApplicationGenerator:

1. Nyl scans the configured directory for YAML files
2. Each file with a NylRelease is discovered
3. An ArgoCD Application is generated for each NylRelease
4. The ApplicationGenerator is replaced with the generated Applications

This pattern is particularly useful for:

• Managing multiple applications in a single repository
• Bootstrapping ArgoCD to manage itself
• Multi-cluster deployments
• Directory-based organization

Getting Started

1. Install the ArgoCD Plugin in your ArgoCD installation
2. Follow the Bootstrapping Guide to set up Nyl with ArgoCD
3. Use ApplicationGenerator to manage applications at scale
4. Review Best Practices for production deployments

Use Cases

Single Application: Use the Nyl plugin directly in ArgoCD Applications to render Nyl manifests from Git.

Multi-Application: Use ApplicationGenerator to scan a directory and automatically create Applications for each discovered release.

Bootstrap Pattern: Use ApplicationGenerator to have ArgoCD manage itself and all applications declaratively.

Helm Integration: Leverage Nyl’s HelmChart resource within ArgoCD to deploy Helm charts with full templating support.

Plugin Installation

The Nyl ArgoCD plugin enables ArgoCD to render Nyl manifests directly from Git repositories. This guide covers plugin installation and configuration.

Prerequisites

• ArgoCD 2.4+ installed in your cluster
• Access to modify ArgoCD’s configuration
• Nyl binary available in the ArgoCD repo-server

Installation Methods

Method 1: Custom ArgoCD Image (Recommended)

Build a custom ArgoCD repo-server image with Nyl included:

FROM quay.io/argoproj/argocd:v2.9.3

# Switch to root to install Nyl
USER root

# Download and install Nyl
RUN curl -L https://github.com/NiklasRosenstein/nyl/releases/download/vX.Y.Z/nyl-linux-amd64 \
    -o /usr/local/bin/nyl && \
    chmod +x /usr/local/bin/nyl

# Switch back to argocd user
USER argocd

Deploy this custom image by updating the argocd-repo-server deployment:

spec:
  template:
    spec:
      containers:
      - name: argocd-repo-server
        image: your-registry/argocd-with-nyl:v2.9.3

Method 2: Init Container

Use an init container to download Nyl into a shared volume:

spec:
  template:
    spec:
      initContainers:
      - name: install-nyl
        image: alpine:3.18
        command:
        - sh
        - -c
        - |
          apk add --no-cache curl
          curl -L https://github.com/NiklasRosenstein/nyl/releases/download/vX.Y.Z/nyl-linux-amd64 \
            -o /plugins/nyl
          chmod +x /plugins/nyl
        volumeMounts:
        - name: plugins
          mountPath: /plugins
      containers:
      - name: argocd-repo-server
        volumeMounts:
        - name: plugins
          mountPath: /usr/local/bin/nyl
          subPath: nyl
      volumes:
      - name: plugins
        emptyDir: {}

Plugin Configuration

Configure the Nyl plugin in the ArgoCD ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  configManagementPlugins: |
    - name: nyl
      generate:
        command: ["/bin/sh", "-c"]
        args:
        - |
          # Render manifests with Nyl
          nyl render .

Verification

Test the plugin installation:

1. Create a test Application using the Nyl plugin:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: test-nyl
  namespace: argocd
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/your-repo
    targetRevision: HEAD
    path: path/to/nyl/manifests
    plugin:
      name: nyl
  destination:
    server: https://kubernetes.default.svc
    namespace: default

2. Check the Application status:

kubectl get application test-nyl -n argocd
argocd app get test-nyl

3. Verify manifest rendering:

argocd app manifests test-nyl

Passing Environment Variables

You can pass environment variables to the Nyl plugin for configuration:

source:
  plugin:
    name: nyl
    env:
    - name: NYL_RELEASE_NAME
      value: my-app
    - name: NYL_RELEASE_NAMESPACE
      value: production

Troubleshooting

Plugin Not Found

If ArgoCD reports “plugin not found”, check:

1. The plugin name matches exactly: nyl
2. The argocd-cm ConfigMap is in the argocd namespace
3. The repo-server pods have been restarted after configuration changes

kubectl rollout restart deployment argocd-repo-server -n argocd

Command Not Found

If the plugin fails with “nyl: command not found”:

1. Verify Nyl binary is in the PATH: /usr/local/bin/nyl
2. Check file permissions (should be executable)
3. Test manually in the repo-server pod:

kubectl exec -it deployment/argocd-repo-server -n argocd -- nyl --version

Profile Not Found

If Nyl reports “Profile ‘default’ not found”:

1. Ensure your repository contains nyl-profiles.yaml
2. Check the profile name matches what you’re referencing
3. Verify the file is in the repository root or search path

Next Steps

• Bootstrap ArgoCD with Nyl
• Use ApplicationGenerator
• Best Practices

Bootstrapping ArgoCD with Nyl

This guide demonstrates how to bootstrap ArgoCD using Nyl with a self-hosting pattern, where ArgoCD manages itself and all applications through the ApplicationGenerator resource.

Overview

The bootstrap pattern works as follows:

1. Manually apply a bootstrap manifest to install ArgoCD
2. The bootstrap includes an ArgoCD Application pointing to apps.yaml
3. ArgoCD syncs the “apps” Application, running nyl render apps.yaml
4. Nyl sees ApplicationGenerator in apps.yaml and scans for NylRelease files
5. Nyl generates ArgoCD Applications for each found NylRelease
6. ArgoCD receives and creates all child Applications
7. ArgoCD now manages all applications (including itself) declaratively

Prerequisites

• kubectl configured with cluster access
• Git repository for storing manifests
• Basic understanding of ArgoCD concepts

Directory Structure

Organize your repository as follows:

gitops-repo/
├── bootstrap.yaml              # Initial bootstrap (apply once manually)
├── apps.yaml                   # ApplicationGenerator definition
├── argocd/                     # ArgoCD installation manifests
│   └── argocd.yaml            # HelmChart for ArgoCD
├── clusters/
│   └── default/               # Applications for default cluster
│       ├── app1.yaml          # Application 1 with NylRelease
│       ├── app2.yaml          # Application 2 with NylRelease
│       └── app3.yaml          # Application 3 with NylRelease
├── nyl-project.yaml           # Nyl project configuration
├── nyl-profiles.yaml          # Nyl profiles (environments)
└── nyl-secrets.yaml           # Secrets configuration

Step 1: Create Configuration Files

nyl-project.yaml

{}

nyl-profiles.yaml

default:
  values: {}

nyl-secrets.yaml

type: null

Step 2: Create ArgoCD Installation Manifest

Create argocd/argocd.yaml:

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: argocd
  namespace: argocd
---
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: argocd
  namespace: argocd
spec:
  chart:
    repository: https://argoproj.github.io/argo-helm
    name: argo-cd
    version: "5.51.6"
  release:
    name: argocd
    namespace: argocd
    createNamespace: true
  values:
    # ArgoCD configuration
    server:
      extraArgs:
        - --insecure  # For testing; use TLS in production

    # Configure the Nyl plugin
    repoServer:
      volumes:
        - name: plugins
          emptyDir: {}

      initContainers:
        - name: install-nyl
          image: alpine:3.18
          command: [sh, -c]
          args:
            - |
              apk add --no-cache curl
              curl -L https://github.com/NiklasRosenstein/nyl/releases/latest/download/nyl-linux-amd64 \
                -o /plugins/nyl
              chmod +x /plugins/nyl
          volumeMounts:
            - name: plugins
              mountPath: /plugins

      volumeMounts:
        - name: plugins
          mountPath: /usr/local/bin/nyl
          subPath: nyl

    # Plugin configuration
    configs:
      cm:
        configManagementPlugins: |
          - name: nyl
            generate:
              command: ["/bin/sh", "-c"]
              args:
                - nyl render .

Step 3: Create ApplicationGenerator

Create apps.yaml with the ApplicationGenerator resource:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: cluster-apps
  namespace: argocd
spec:
  # Where to create generated Applications
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd

  # Source repository configuration
  source:
    repoURL: https://github.com/your-org/gitops-repo.git
    targetRevision: HEAD
    path: clusters/default

    # Optional: file filtering
    include:
      - "*.yaml"
      - "*.yml"
    exclude:
      - ".*"           # Skip hidden files
      - "_*"           # Skip files starting with underscore
      - "apps.yaml"    # Avoid recursion

  # ArgoCD project for generated Applications
  project: default

  # Default sync policy for all generated Applications
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

  # Labels added to all generated Applications
  labels:
    managed-by: nyl
    generator: cluster-apps

  # Annotations added to all generated Applications
  annotations:
    docs-url: https://wiki.example.com/cluster-apps

Step 4: Create Sample Applications

Create clusters/default/nginx.yaml:

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: nginx
  namespace: default
---
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: nginx
spec:
  chart:
    repository: https://charts.bitnami.com/bitnami
    name: nginx
    version: "15.4.4"
  release:
    name: nginx
    namespace: default
  values:
    replicaCount: 2
    service:
      type: ClusterIP

Create clusters/default/redis.yaml:

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: redis
  namespace: default
---
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: redis
spec:
  chart:
    repository: https://charts.bitnami.com/bitnami
    name: redis
    version: "18.4.0"
  release:
    name: redis
    namespace: default
  values:
    architecture: standalone
    auth:
      enabled: false

Step 5: Create Bootstrap Manifest

Create bootstrap.yaml:

# Namespace for ArgoCD
apiVersion: v1
kind: Namespace
metadata:
  name: argocd
---
# Application to install ArgoCD itself
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: argocd
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/gitops-repo.git
    targetRevision: HEAD
    path: argocd
    plugin:
      name: nyl
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
---
# Application generator that creates all other Applications
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: apps
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-org/gitops-repo.git
    targetRevision: HEAD
    path: .
    plugin:
      name: nyl
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

Important: In the apps Application, the path is . (repository root) because apps.yaml contains the ApplicationGenerator which will scan clusters/default.

Step 6: Bootstrap the Cluster

1. Commit all files to your Git repository:

git add .
git commit -m "Initial Nyl + ArgoCD bootstrap"
git push origin main

2. Apply the bootstrap manifest to your cluster:

kubectl apply -f bootstrap.yaml

3. Wait for ArgoCD to install:

kubectl wait --for=condition=available --timeout=5m \
  deployment/argocd-server -n argocd

4. Access ArgoCD UI:

# Port forward to access UI
kubectl port-forward svc/argocd-server -n argocd 8080:443

# Get admin password
kubectl get secret argocd-initial-admin-secret -n argocd \
  -o jsonpath='{.data.password}' | base64 -d

5. Verify Applications were created:

kubectl get applications -n argocd

Expected output:

NAME     SYNC STATUS   HEALTH STATUS
argocd   Synced        Healthy
apps     Synced        Healthy
nginx    Synced        Healthy
redis    Synced        Healthy

How It Works

When you apply bootstrap.yaml:

1. ArgoCD Namespace Created: The argocd namespace is created
2. ArgoCD Application Syncs: ArgoCD installs itself via the Nyl plugin
3. Apps Application Syncs: The “apps” Application runs nyl render .
4. ApplicationGenerator Processes: Nyl finds apps.yaml, sees ApplicationGenerator
5. Directory Scanned: Nyl scans clusters/default/ for YAML files
6. Applications Generated: For each NylRelease found (nginx, redis), Nyl generates an ArgoCD Application
7. Applications Created: ArgoCD receives the generated Applications and creates them
8. Child Applications Sync: Each generated Application syncs its resources to the cluster

Verification

Check Application Status

# View all applications
argocd app list

# Get details of a specific application
argocd app get nginx

# View application tree
argocd app get nginx --show-operation

# View rendered manifests
argocd app manifests nginx

Verify Nyl Plugin

# Check if Nyl is available in repo-server
kubectl exec -it deployment/argocd-repo-server -n argocd -- nyl --version

# View plugin configuration
kubectl get configmap argocd-cm -n argocd -o yaml | grep -A10 configManagementPlugins

View Generated Applications

# See what ApplicationGenerator produced
kubectl exec -it deployment/argocd-repo-server -n argocd -- \
  sh -c 'cd /tmp && git clone https://github.com/your-org/gitops-repo.git && \
  cd gitops-repo && nyl render apps.yaml'

Adding New Applications

To add a new application to be managed by ArgoCD:

1. Create a new YAML file in clusters/default/:

# clusters/default/postgres.yaml
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: postgres
  namespace: database
---
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: postgres
spec:
  chart:
    repository: https://charts.bitnami.com/bitnami
    name: postgresql
    version: "13.2.24"
  release:
    name: postgres
    namespace: database
    createNamespace: true
  values:
    auth:
      username: myuser
      database: mydb

2. Commit and push to Git:

git add clusters/default/postgres.yaml
git commit -m "Add PostgreSQL application"
git push origin main

3. ArgoCD automatically detects the change and creates the postgres Application

No manual intervention needed! The ApplicationGenerator pattern discovers new applications automatically.

Troubleshooting

Applications Not Created

If generated Applications don’t appear:

1. Check the “apps” Application status:

   argocd app get apps
   

2. View the rendered manifests:

   argocd app manifests apps
   

3. Check if ApplicationGenerator is processing correctly:

   # Should show ArgoCD Applications, not the ApplicationGenerator
   argocd app manifests apps | grep "kind: Application"
   

Plugin Failures

If the Nyl plugin fails:

1. Check repo-server logs:

   kubectl logs deployment/argocd-repo-server -n argocd -f
   

2. Verify Nyl installation:

   kubectl exec deployment/argocd-repo-server -n argocd -- which nyl
   kubectl exec deployment/argocd-repo-server -n argocd -- nyl --version
   

3. Test Nyl render manually:

   kubectl exec -it deployment/argocd-repo-server -n argocd -- sh
   # Inside the pod:
   cd /tmp
   git clone <your-repo>
   cd <repo-name>
   nyl render apps.yaml
   

Sync Issues

If Applications fail to sync:

1. Check Application health:

   argocd app get <app-name>
   

2. View sync operation details:

   argocd app get <app-name> --show-operation
   

3. Force refresh and sync:

   argocd app sync <app-name> --force
   

Next Steps

• ApplicationGenerator Reference - Detailed field documentation
• Best Practices - Production recommendations
• Multi-Cluster Setup - Managing multiple clusters

ApplicationGenerator Reference

The ApplicationGenerator resource enables automatic discovery and generation of ArgoCD Applications from NylRelease files in a Git repository directory.

Note: Git repositories are cloned automatically by Nyl. You don’t need to manually clone repositories. See the Git Integration guide for cache management and configuration details.

Resource Definition

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: string              # Generator name (optional)
  namespace: string         # Namespace (optional, typically 'argocd')
spec:
  destination:              # Required
    server: string          # Kubernetes API server URL
    namespace: string       # Namespace for generated Applications
  source:                   # Required
    repoURL: string         # Git repository URL
    targetRevision: string  # Branch, tag, or commit (default: "HEAD")
    path: string            # Directory to scan
    include: [string]       # Include patterns (default: ["*.yaml", "*.yml"])
    exclude: [string]       # Exclude patterns (default: [".*", "_*"])
  project: string           # ArgoCD project (default: "default")
  syncPolicy:               # Optional sync policy for generated Applications
    automated:
      prune: bool
      selfHeal: bool
    syncOptions: [string]
  applicationNameTemplate: string  # Naming template (default: "{{ .release.name }}")
  labels: {string: string}         # Labels for generated Applications
  annotations: {string: string}    # Annotations for generated Applications

Field Reference

metadata

Standard Kubernetes metadata for the ApplicationGenerator resource itself.

• name (optional): Identifier for this generator
• namespace (optional): Namespace where this resource lives (typically argocd)

spec.destination

Defines where generated Applications should be created and what cluster they target.

• server (required): Kubernetes API server URL for the target cluster

  • Example: https://kubernetes.default.svc (in-cluster)
  • Example: https://my-cluster.example.com:6443 (external cluster)

• namespace (required): Namespace where generated Applications are created

  • Typically argocd
  • Must match where ArgoCD is installed

spec.source

Configures the Git repository and directory scanning behavior.

• repoURL (required): Git repository URL

  • HTTPS: https://github.com/org/repo.git
  • SSH: git@github.com:org/repo.git

• targetRevision (optional, default: "HEAD"): Git reference to use

  • Branch: main, develop
  • Tag: v1.0.0
  • Commit: abc123def456

• path (required): Directory path to scan for YAML files

  • Relative to repository root
  • Example: clusters/production
  • Example: apps

• include (optional, default: ["*.yaml", "*.yml"]): Glob patterns for files to include

  • Supports simple glob syntax: *.yaml, *.yml, app*.yaml
  • Multiple patterns are OR’d together

• exclude (optional, default: [".*", "_*"]): Glob patterns for files to exclude

  • Takes precedence over include patterns
  • Default excludes hidden files (.) and underscore-prefixed files (_)
  • Example: ["test_*", ".*", "backup*"]

spec.project

• project (optional, default: "default"): ArgoCD project name for generated Applications
  • Must be an existing ArgoCD AppProject
  • Used for RBAC and resource restrictions

spec.syncPolicy

Optional default sync policy applied to all generated Applications.

• automated (optional): Enable automated sync

  • prune (bool): Delete resources no longer defined in Git
  • selfHeal (bool): Force resource state to match Git

• syncOptions (optional): List of sync options

  • CreateNamespace=true: Create destination namespace if missing
  • PruneLast=true: Prune resources after other operations
  • RespectIgnoreDifferences=true: Respect ignore differences
  • ApplyOutOfSyncOnly=true: Only apply resources that are out of sync

spec.applicationNameTemplate

• applicationNameTemplate (optional, default: "{{ .release.name }}"): Template for Application names
  • Currently supports: {{ .release.name }}, {{ .release.namespace }}
  • Future: Full Handlebars/Tera template support

spec.labels

Key-value map of labels to add to all generated Applications.

Example:

labels:
  managed-by: nyl
  team: platform
  environment: production

spec.annotations

Key-value map of annotations to add to all generated Applications.

Example:

annotations:
  docs-url: https://wiki.example.com/apps
  team-slack: "#platform-team"

File Filtering

The ApplicationGenerator scans the configured directory and applies include/exclude patterns:

Pattern Matching

Patterns use simple glob syntax:

• *.yaml - Matches files ending with .yaml
• *.yml - Matches files ending with .yml
• app* - Matches files starting with app
• .* - Matches hidden files (starting with .)
• _* - Matches files starting with underscore
• test_*.yaml - Matches test_*.yaml files
• Exact match: apps.yaml - Matches only apps.yaml

Filtering Logic

1. File must be a regular file (not directory)
2. File must match at least one include pattern
3. File must NOT match any exclude pattern
4. Nyl parses the file and looks for a NylRelease resource
5. If NylRelease found, an Application is generated

Default Patterns

By default:

• Include: ["*.yaml", "*.yml"] - All YAML files
• Exclude: [".*", "_*"] - Hidden files and underscore-prefixed files

This prevents accidental inclusion of:

• Hidden files like .secrets.yaml, .git/
• Backup files like _backup.yaml
• Test files like _test_app.yaml

Examples

Basic Usage

Simplest ApplicationGenerator for scanning a directory:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: my-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: apps

This scans apps/ for *.yaml and *.yml files, generates an Application for each NylRelease found.

With Automated Sync

Enable automatic synchronization and pruning:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: production-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    targetRevision: production
    path: clusters/production
  project: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true
      - PruneLast=true

Custom File Filtering

Include only specific files and exclude test files:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: filtered-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: apps
    include:
      - "prod-*.yaml"
      - "core-*.yaml"
    exclude:
      - ".*"
      - "_*"
      - "test-*"
      - "*-backup.yaml"

With Labels and Annotations

Add metadata to generated Applications:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: team-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: teams/platform/apps
  labels:
    team: platform
    managed-by: nyl
    environment: production
  annotations:
    team-slack: "#platform-team"
    oncall-pagerduty: "P123ABC"

Multi-Cluster Setup

Generate Applications for an external cluster:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: staging-cluster-apps
  namespace: argocd
spec:
  destination:
    server: https://staging-cluster.example.com:6443
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    targetRevision: main
    path: clusters/staging
  project: staging
  syncPolicy:
    automated:
      selfHeal: true

Note: The cluster must be registered in ArgoCD first.

Multiple Generators

You can have multiple ApplicationGenerators in the same file:

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: core-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: core
  project: core
---
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: addon-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: addons
  project: addons

Generated Application Structure

For a NylRelease file like this:

# clusters/default/nginx.yaml
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: nginx
  namespace: web
---
# ... other resources ...

The ApplicationGenerator produces:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: nginx                    # From NylRelease.metadata.name
  namespace: argocd              # From generator.spec.destination.namespace
  labels:                        # From generator.spec.labels
    managed-by: nyl
  annotations:                   # From generator.spec.annotations
    docs-url: https://wiki.example.com/apps
spec:
  project: default               # From generator.spec.project
  source:
    repoURL: https://github.com/myorg/gitops.git  # From generator.spec.source.repoURL
    path: clusters/default       # Directory of the file
    targetRevision: HEAD         # From generator.spec.source.targetRevision
    plugin:
      name: nyl
      env:
        - name: NYL_RELEASE_NAME
          value: nginx
        - name: NYL_RELEASE_NAMESPACE
          value: web
  destination:
    server: https://kubernetes.default.svc  # From generator.spec.destination.server
    namespace: web               # From NylRelease.metadata.namespace
  syncPolicy:                    # From generator.spec.syncPolicy
    automated:
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

Behavior

Processing Flow

1. When nyl render encounters an ApplicationGenerator resource:

   • The ApplicationGenerator is extracted and NOT included in output
   • The source path is scanned for YAML files
   • Files are filtered by include/exclude patterns
   • Each file is parsed for a NylRelease resource
   • An ArgoCD Application is generated for each NylRelease
   • Generated Applications are added to the output

2. The ApplicationGenerator acts as an “offline controller” pattern:

   • Processing happens during nyl render
   • No runtime controller or operator needed
   • Deterministic output (same input always produces same output)
   • Works in ArgoCD plugin context

Path Resolution

The source.path in ApplicationGenerator is resolved relative to the base directory:

• If rendering a file: /path/to/apps.yaml → base is /path/to
• If rendering a directory: /path/to/project → base is /path/to/project
• Source path clusters/default → scans /path/to/clusters/default

For generated Applications:

• The path field points to the directory containing the NylRelease file
• Relative to the repository root

Validation

ApplicationGenerator is validated when parsed:

• spec.destination.server must not be empty
• spec.destination.namespace must not be empty
• spec.source.repoURL must not be empty
• spec.source.path must not be empty

Invalid ApplicationGenerator resources cause nyl render to fail with a clear error message.

Limitations (Phase 1)

Current limitations:

• No Git cloning: Repositories must be pre-cloned (suitable for ArgoCD plugin use)
• Simple glob patterns: No full glob library support (e.g., **/*.yaml not supported)
• No templating: applicationNameTemplate only supports basic substitution
• Single repository: Cannot scan multiple Git repositories in one generator

These limitations are addressed in Phase 2 (future enhancement).

Next Steps

• Bootstrapping Guide - Step-by-step setup
• Best Practices - Production recommendations

Repository Secrets

Nyl automatically discovers credentials from ArgoCD repository secrets to authenticate with private Git repositories. This enables seamless integration with private Helm charts and GitOps repositories.

Overview

When Nyl encounters a Git URL in a HelmChart or ApplicationGenerator resource, it:

1. Queries ArgoCD repository secrets from the argocd namespace
2. Matches the Git URL to an appropriate secret
3. Uses the discovered credentials for authentication
4. Falls back to SSH agent authentication if no secret matches

This approach provides:

• Zero configuration: Credentials are discovered automatically from existing ArgoCD secrets
• Security: Credentials remain in Kubernetes secrets, never in configuration files
• Consistency: Same credentials used by ArgoCD and Nyl
• Flexibility: Supports both SSH and HTTPS authentication

Secret Types

ArgoCD supports two types of repository credential secrets, both of which are supported by Nyl:

Repository Secrets (Fine-Grained)

Repository secrets (argocd.argoproj.io/secret-type=repository) provide credentials for specific repositories using exact URL matching.

Example:

apiVersion: v1
kind: Secret
metadata:
  name: myorg-charts-repo
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
type: Opaque
stringData:
  type: git  # Only "git" type secrets are used (defaults to "git" if omitted)
  url: git@github.com:myorg/charts.git
  sshPrivateKey: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ...
    -----END OPENSSH PRIVATE KEY-----

Use when:

• Managing credentials for individual repositories
• Need fine-grained access control
• Different credentials for different repos on same host

Repository Credential Templates (Scalable)

Repository credential templates (argocd.argoproj.io/secret-type=repo-creds) provide credentials for multiple repositories matching a URL pattern.

Example:

apiVersion: v1
kind: Secret
metadata:
  name: github-myorg-all
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repo-creds
type: Opaque
stringData:
  type: git  # Only "git" type secrets are used (defaults to "git" if omitted)
  url: https://github.com/myorg/*
  username: myusername
  password: ghp_token123

Pattern Examples:

• https://github.com/myorg/* - All repos in organization
• https://github.com/* - All GitHub repos
• git@gitlab.com:*/* - All GitLab repos with org and repo

Use when:

• Managing many repositories from same provider
• Centralized credential management
• Single credential applies to multiple repos

Important: Both secret types must have type: git (or omit the type field, which defaults to git). Secrets with other types (e.g., helm, oci) are ignored.

Authentication Methods

SSH Key Authentication

SSH key authentication is the recommended method for private repositories.

Creating an SSH repository secret:

# Create secret with SSH private key
kubectl create secret generic my-private-repo \
  -n argocd \
  --from-literal=url=git@github.com:myorg/charts.git \
  --from-file=sshPrivateKey=$HOME/.ssh/id_rsa

# Label the secret for ArgoCD
kubectl label secret my-private-repo \
  -n argocd \
  argocd.argoproj.io/secret-type=repository

Secret structure:

apiVersion: v1
kind: Secret
metadata:
  name: my-private-repo
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
type: Opaque
data:
  url: Z2l0QGdpdGh1Yi5jb206bXlvcmcvY2hhcnRzLmdpdA==  # base64: git@github.com:myorg/charts.git
  sshPrivateKey: LS0tLS1CRUdJTi...                    # base64 encoded SSH private key

HTTPS Token Authentication

HTTPS authentication uses personal access tokens or passwords.

Creating an HTTPS repository secret:

# Create secret with username and token
kubectl create secret generic my-private-repo-https \
  -n argocd \
  --from-literal=url=https://github.com/myorg/charts.git \
  --from-literal=username=myusername \
  --from-literal=password=ghp_your_token_here

# Label the secret for ArgoCD
kubectl label secret my-private-repo-https \
  -n argocd \
  argocd.argoproj.io/secret-type=repository

Secret structure:

apiVersion: v1
kind: Secret
metadata:
  name: my-private-repo-https
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
type: Opaque
data:
  url: aHR0cHM6Ly9naXRodWIuY29tL215b3JnL2NoYXJ0cy5naXQ=  # base64: https://github.com/myorg/charts.git
  username: bXl1c2VybmFtZQ==                              # base64: myusername
  password: Z2hwX3lvdXJfdG9rZW5faGVyZQ==                  # base64: ghp_your_token_here

SSH Agent Fallback

If no ArgoCD secret matches a Git URL, Nyl automatically falls back to SSH agent authentication for SSH URLs:

# Start SSH agent
eval "$(ssh-agent -s)"

# Add your SSH key
ssh-add ~/.ssh/id_rsa

# Nyl will now use the SSH agent for authentication
nyl render my-chart.yaml

URL Matching

Nyl matches Git URLs to repository secrets using multiple strategies with specific precedence rules.

Credential Selection Precedence

When multiple secrets could apply to a repository, Nyl uses this precedence:

1. Exact match (repository secret with exact URL match)
2. Pattern match (repo-creds secret with matching pattern)
3. Hostname fallback (repository secret with same hostname)

Example:

Given these secrets:

# 1. Exact match (highest precedence)
apiVersion: v1
kind: Secret
metadata:
  name: charts-exact
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: https://github.com/myorg/charts.git
  sshPrivateKey: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ...specific key for charts repo...

# 2. Pattern match (medium precedence)
apiVersion: v1
kind: Secret
metadata:
  name: myorg-pattern
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repo-creds
stringData:
  url: https://github.com/myorg/*
  username: myuser
  password: ghp_token_for_myorg

# 3. Hostname fallback (lowest precedence)
apiVersion: v1
kind: Secret
metadata:
  name: github-fallback
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  url: https://github.com/other/repo.git
  username: myuser
  password: ghp_generic_token

For URL https://github.com/myorg/charts.git:

• Uses exact match (#1) ✅

For URL https://github.com/myorg/other-repo.git:

• No exact match
• Uses pattern match (#2) ✅

For URL https://github.com/different-org/repo.git:

• No exact match
• No pattern match
• Uses hostname fallback (#3) ✅

Exact Match

The secret’s url field exactly matches the requested URL (after normalization):

# Secret URL (repository type)
url: https://github.com/myorg/charts.git

# Matches
- https://github.com/myorg/charts.git
- https://github.com/myorg/charts      # .git suffix optional

Pattern Match

The secret’s url field contains wildcards (*) that match the requested URL:

# Secret URL (repo-creds type)
url: https://github.com/myorg/*

# Matches
- https://github.com/myorg/charts.git
- https://github.com/myorg/other-repo.git
- https://github.com/myorg/any-repo

# Does NOT match
- https://github.com/otherorg/repo.git  # Different organization
- https://gitlab.com/myorg/repo.git     # Different host

Pattern Examples:

# Organization-level access
url: https://github.com/myorg/*
# Matches: https://github.com/myorg/repo1, https://github.com/myorg/repo2

# Host-level access (all repos on GitHub)
url: https://github.com/*
# Matches: https://github.com/anyorg/anyrepo

# SSH with pattern
url: git@github.com:myorg/*
# Matches: git@github.com:myorg/charts, git@github.com:myorg/other

# Multi-level wildcard
url: https://gitlab.com/*/*
# Matches: https://gitlab.com/group/project, https://gitlab.com/org/repo

Hostname Fallback

If no exact or pattern match is found, Nyl matches by hostname. This allows a single repository secret to authenticate all repositories on the same host:

# Secret URL (repository type)
url: https://github.com/myorg/charts.git

# Also matches (same hostname, no exact/pattern match)
- https://github.com/myorg/other-repo.git
- https://github.com/another-org/repo.git

URL Normalization

URLs are normalized before matching:

• Case-insensitive comparison
• .git suffix is optional
• SSH shorthand converted to full URL:
  • git@github.com:org/repo → ssh://git@github.com/org/repo
• Trailing slashes removed

This means these URLs are considered equivalent:

• https://GitHub.com/MyOrg/Repo.git
• https://github.com/myorg/repo
• git@github.com:myorg/repo (when matching SSH patterns)

Examples

Private Helm Chart

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: my-app
spec:
  chart:
    git: git@github.com:myorg/private-charts.git
    git_ref: main
    path: charts/my-app
  release:
    name: my-app
    namespace: default

Nyl will:

1. Look for ArgoCD secret with URL git@github.com:myorg/private-charts.git
2. If not found, look for secret with hostname github.com
3. If not found, try SSH agent
4. Clone repository and extract chart from charts/my-app

Private ApplicationGenerator

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: microservices
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops-repo.git
    targetRevision: main
    path: apps

Nyl will:

1. Look for ArgoCD secret with URL https://github.com/myorg/gitops-repo.git
2. If found, use credentials to clone repository
3. Scan apps/ directory for application definitions
4. Generate ArgoCD Application resources

Troubleshooting

Authentication Failures

Error: Authentication failed for git@github.com:myorg/repo.git: Failed to use SSH agent

Solutions:

1. Create an ArgoCD repository secret with SSH key
2. Start SSH agent and add your key: ssh-add ~/.ssh/id_rsa
3. Verify SSH key has access to the repository

Error: No credentials found for repository: https://github.com/myorg/repo.git

Solutions:

1. Create an ArgoCD repository secret for this URL
2. Verify secret has label argocd.argoproj.io/secret-type=repository
3. Check secret is in argocd namespace

Permission Issues

Error: Failed to query ArgoCD secrets: Forbidden

Solutions:

1. Verify Nyl has RBAC permissions to read secrets in argocd namespace
2. Add ClusterRole/Role with get, list permissions on secrets
3. Bind role to Nyl service account

URL Matching Issues

If credentials aren’t being discovered:

1. Check URL format matches between secret and chart
2. Verify secret URL is normalized (lowercase, no trailing slash)
3. Test with exact URL match before relying on hostname fallback
4. Check secret has the correct label

Debugging

Enable debug logging to see credential discovery:

RUST_LOG=debug nyl render my-chart.yaml

Look for log messages:

• Discovering credentials from ArgoCD secrets
• Found credential for URL: ...
• No credential found, falling back to SSH agent

Best Practices

1. Use SSH keys: Preferred over HTTPS tokens for better security
2. Choose the right secret type:
   • Use repo-creds for organization-wide access (scalable)
   • Use repository for specific repos needing different credentials
   • Combine both types for flexibility with proper precedence
3. Pattern-based credentials: Use repo-creds with patterns like https://github.com/myorg/* to centrally manage credentials for multiple repositories
4. Leverage precedence: Use exact match repository secrets to override broader repo-creds patterns for specific repos
5. Rotate credentials: Update secrets regularly and test after rotation
6. Least privilege: Ensure SSH keys/tokens have minimal required permissions
7. Monitor access: Review ArgoCD secret access logs periodically
8. Document secrets: Maintain inventory of repository secrets, their patterns, and their purpose

Example Setup:

# 1. Organization-wide access (repo-creds)
apiVersion: v1
kind: Secret
metadata:
  name: github-myorg-default
  labels:
    argocd.argoproj.io/secret-type: repo-creds
stringData:
  type: git
  url: https://github.com/myorg/*
  username: ci-bot
  password: ghp_org_token

---
# 2. Specific repo with different credentials (repository)
apiVersion: v1
kind: Secret
metadata:
  name: sensitive-repo-override
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: git
  url: https://github.com/myorg/sensitive-repo.git
  sshPrivateKey: |
    -----BEGIN OPENSSH PRIVATE KEY-----
    ...dedicated SSH key for sensitive repo...

This setup provides:

• Default credentials for all myorg repositories via pattern match
• Override with dedicated SSH key for sensitive-repo via exact match
• Separation of concerns and least privilege access

Kubernetes RBAC

Nyl requires these permissions to discover repository secrets:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nyl-secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]
  # Optionally scope to argocd namespace
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: nyl-secret-reader
subjects:
- kind: ServiceAccount
  name: nyl
  namespace: argocd
roleRef:
  kind: ClusterRole
  name: nyl-secret-reader
  apiGroup: rbac.authorization.k8s.io

Security Considerations

• Credential storage: Secrets are stored in Kubernetes, encrypted at rest
• Access control: RBAC limits who can read repository secrets
• Credential scope: Each secret applies only to specific repositories
• No plaintext: Credentials never appear in logs or configuration files
• SSH keys: Use dedicated deploy keys with read-only access
• Token rotation: Update tokens/keys regularly and monitor for leaks

Best Practices

This guide covers recommended patterns and best practices for using Nyl with ArgoCD in production environments.

Directory Structure

Recommended Layout

Organize your GitOps repository with clear separation of concerns:

gitops-repo/
├── bootstrap/
│   └── bootstrap.yaml          # Initial bootstrap manifest
├── argocd/
│   └── argocd.yaml            # ArgoCD installation
├── apps.yaml                   # ApplicationGenerator for apps
├── clusters/
│   ├── production/
│   │   ├── app1.yaml
│   │   ├── app2.yaml
│   │   └── app3.yaml
│   ├── staging/
│   │   ├── app1.yaml
│   │   └── app2.yaml
│   └── development/
│       └── app1.yaml
├── base/                       # Shared base configurations
│   ├── postgres/
│   └── redis/
├── nyl-project.yaml
├── nyl-profiles.yaml
└── nyl-secrets.yaml

Directory Organization Strategies

By Environment:

clusters/
├── production/
├── staging/
└── development/

• Clear separation of environments
• Easy to apply different policies per environment
• Suitable for ApplicationGenerator scanning

By Team:

teams/
├── platform/
├── data/
└── ml/

• Delegation to team-specific directories
• Team-level RBAC with ArgoCD Projects
• Enables team autonomy

By Application Type:

├── core-services/
├── data-services/
├── monitoring/
└── security/

• Logical grouping by function
• Useful for dependency management
• Clear categorization

Hybrid Approach (Recommended):

├── core/                       # Critical platform services
│   ├── argocd/
│   ├── cert-manager/
│   └── ingress-nginx/
├── clusters/
│   ├── production/
│   │   ├── team-a/
│   │   └── team-b/
│   └── staging/
│       ├── team-a/
│       └── team-b/
└── shared/                     # Shared configurations
    ├── postgres/
    └── redis/

ApplicationGenerator Patterns

One Generator Per Environment

Create separate ApplicationGenerators for each environment:

# apps-production.yaml
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: production-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    targetRevision: main
    path: clusters/production
  project: production
  syncPolicy:
    automated:
      prune: true
      selfHeal: true
  labels:
    environment: production
---
# apps-staging.yaml
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: staging-apps
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    targetRevision: main
    path: clusters/staging
  project: staging
  syncPolicy:
    automated:
      selfHeal: true  # Note: No prune in staging
  labels:
    environment: staging

One App Per File

Each application should have its own YAML file:

clusters/production/
├── nginx.yaml          # One app
├── postgres.yaml       # One app
└── redis.yaml          # One app

Benefits:

• Clear ownership and Git history per app
• Easy to move apps between environments
• Better for code review
• Simpler conflict resolution

Avoid putting multiple apps in one file (unless they’re tightly coupled).

Exclude Patterns

Use exclude patterns to prevent accidental Application generation:

spec:
  source:
    exclude:
      - ".*"              # Hidden files
      - "_*"              # Underscore prefix (templates, WIP)
      - "*.backup"        # Backup files
      - "test-*"          # Test files
      - "apps.yaml"       # The generator itself
      - "README.md"       # Documentation

Sync Policies

Production Environments

For production, be conservative:

syncPolicy:
  automated:
    prune: true         # Delete removed resources
    selfHeal: true      # Force Git state
  syncOptions:
    - CreateNamespace=true
    - PruneLast=true    # Prune after other operations

Why?

• prune: true: Ensures removed manifests are deleted
• selfHeal: true: Prevents manual changes (drift detection)
• PruneLast: Safer deletion order

Staging/Development Environments

For non-production, allow more flexibility:

syncPolicy:
  automated:
    prune: false        # Don't auto-delete (allow manual testing)
    selfHeal: true      # Still enforce Git state

Why?

• prune: false: Allows temporary manual resources for testing
• selfHeal: true: Still prevents accidental drift

Manual Sync for Critical Services

For very critical services (databases, auth), consider manual sync:

# No automated syncPolicy
# Sync manually via ArgoCD UI or CLI

Secret Management

Option 1: Sealed Secrets

Use Bitnami Sealed Secrets for encrypted secrets in Git:

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: app
  namespace: default
---
apiVersion: v1
kind: SealedSecret
metadata:
  name: app-secrets
  namespace: default
spec:
  encryptedData:
    password: AgBjW8X... # Encrypted data safe for Git

Option 2: External Secrets Operator

Sync secrets from external providers (AWS Secrets Manager, Vault, etc.):

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-secrets
  namespace: default
spec:
  secretStoreRef:
    name: aws-secrets-manager
  target:
    name: app-secrets
  data:
    - secretKey: password
      remoteRef:
        key: /app/password

Option 3: Nyl Secrets (SOPS)

Use Nyl’s built-in SOPS integration:

# nyl-secrets.yaml
type: sops
path: ./.secrets.yaml

# .secrets.yaml (encrypted with SOPS)
database:
  password: ENC[AES256_GCM,data:xxx,iv:yyy,tag:zzz]

Never commit plaintext secrets to Git!

Multi-Cluster Setup

Hub-and-Spoke Model

Manage multiple clusters from a central ArgoCD instance:

# apps-prod-cluster1.yaml
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: prod-cluster1-apps
spec:
  destination:
    server: https://cluster1.example.com:6443
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: clusters/cluster1
  labels:
    cluster: cluster1
---
# apps-prod-cluster2.yaml
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: prod-cluster2-apps
spec:
  destination:
    server: https://cluster2.example.com:6443
    namespace: argocd
  source:
    repoURL: https://github.com/myorg/gitops.git
    path: clusters/cluster2
  labels:
    cluster: cluster2

Register clusters in ArgoCD:

argocd cluster add cluster1-context
argocd cluster add cluster2-context

Per-Cluster ArgoCD

Each cluster has its own ArgoCD instance:

gitops-repo/
├── clusters/
│   ├── prod-us-east/
│   │   ├── argocd/
│   │   ├── apps.yaml          # ApplicationGenerator for this cluster
│   │   └── apps/
│   └── prod-eu-west/
│       ├── argocd/
│       ├── apps.yaml
│       └── apps/

Each cluster bootstraps independently.

Monitoring and Observability

Prometheus Metrics

ArgoCD exports metrics; monitor these key indicators:

• argocd_app_info: Application status
• argocd_app_sync_total: Sync operations
• argocd_app_health_status: Health status

Alerts

Set up alerts for:

# OutOfSync alert
- alert: ArgoCDAppOutOfSync
  expr: argocd_app_sync_status{sync_status="OutOfSync"} == 1
  for: 15m
  labels:
    severity: warning

# Degraded health
- alert: ArgoCDAppUnhealthy
  expr: argocd_app_health_status{health_status!="Healthy"} == 1
  for: 10m
  labels:
    severity: critical

Notifications

Configure ArgoCD notifications for Slack, email, or PagerDuty:

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  service.slack: |
    token: $slack-token
  template.app-sync-status: |
    message: Application {{.app.metadata.name}} sync is {{.app.status.sync.status}}
  trigger.on-sync-failed: |
    - when: app.status.sync.status == 'Failed'
      send: [app-sync-status]

CI/CD Integration

Validation Pipeline

Add CI checks to validate Nyl manifests before merge:

# .github/workflows/validate.yml
name: Validate Manifests
on: [pull_request]
jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install Nyl
        run: |
          curl -L https://github.com/NiklasRosenstein/nyl/releases/latest/download/nyl-linux-amd64 \
            -o /usr/local/bin/nyl
          chmod +x /usr/local/bin/nyl
      - name: Validate
        run: nyl validate .
      - name: Render
        run: nyl render apps.yaml > /dev/null

Dry-Run Rendering

Test ApplicationGenerator output in CI:

# Render and check output
nyl render apps.yaml > output.yaml

# Verify Applications were generated
cat output.yaml | grep "kind: Application" | wc -l

# Check specific apps exist
grep "name: nginx" output.yaml
grep "name: postgres" output.yaml

Preview Environments

Create preview environments for PRs:

# .github/workflows/preview.yml
name: Preview Environment
on: [pull_request]
jobs:
  preview:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Deploy Preview
        run: |
          # Create preview namespace
          kubectl create namespace pr-${{ github.event.pull_request.number }}

          # Render manifests with preview profile
          nyl render -e preview apps.yaml | kubectl apply -n pr-${{ github.event.pull_request.number }} -f -

Performance Considerations

Repository Size

Keep repositories focused:

• Separate infrastructure repo from application repos
• Use Git submodules for shared configurations if needed
• Archive old applications to separate branches

Sync Frequency

Adjust sync frequency based on needs:

# Faster sync for development
metadata:
  annotations:
    argocd.argoproj.io/sync-options: "Timeout=120"

# Slower sync for large apps
metadata:
  annotations:
    argocd.argoproj.io/sync-options: "Timeout=600"

Resource Limits

Set appropriate resource limits for ArgoCD components:

spec:
  values:
    repoServer:
      resources:
        limits:
          cpu: "1000m"
          memory: "1Gi"
        requests:
          cpu: "500m"
          memory: "512Mi"

Disaster Recovery

Backup Strategy

1. Git is Source of Truth: All manifests in Git
2. Backup ArgoCD Configuration:

   kubectl get applications -n argocd -o yaml > argocd-apps-backup.yaml
   kubectl get appprojects -n argocd -o yaml > argocd-projects-backup.yaml
   

3. Cluster Backups: Use Velero or similar for cluster state

Recovery Procedure

1. Restore cluster from backup (if needed)
2. Re-apply bootstrap manifest:

   kubectl apply -f bootstrap.yaml
   

3. ArgoCD recreates all Applications from Git
4. Applications sync and restore workloads

Why this works: Git is the source of truth; ArgoCD recreates everything from Git automatically.

Security

RBAC with Projects

Use ArgoCD Projects for RBAC:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: team-a
  namespace: argocd
spec:
  description: Team A applications
  sourceRepos:
    - https://github.com/myorg/gitops.git
  destinations:
    - namespace: team-a-*
      server: https://kubernetes.default.svc
  clusterResourceWhitelist:
    - group: ''
      kind: Namespace
  namespaceResourceWhitelist:
    - group: '*'
      kind: '*'

Signed Commits

Require GPG-signed commits for production:

spec:
  source:
    repoURL: https://github.com/myorg/gitops.git
    targetRevision: main
  # In ArgoCD settings
  metadata:
    annotations:
      argocd.argoproj.io/verify-signature: "true"

Least Privilege

• ArgoCD service account should have minimal permissions
• Use separate service accounts per ApplicationGenerator/Project
• Audit ArgoCD RBAC regularly

Troubleshooting

Common Issues

Issue: Applications not syncing

• Check: ArgoCD repo-server logs
• Fix: Ensure Nyl plugin is installed and accessible

Issue: Wrong Applications generated

• Check: Render manifests locally: nyl render apps.yaml
• Fix: Adjust include/exclude patterns

Issue: Sync timeouts

• Check: Large Helm charts or slow cluster
• Fix: Increase timeout annotation

Debug Commands

# View ApplicationGenerator output
nyl render apps.yaml

# Check ArgoCD Application
argocd app get <app-name>

# View rendered manifests
argocd app manifests <app-name>

# Force refresh
argocd app get <app-name> --refresh

# View repo-server logs
kubectl logs deployment/argocd-repo-server -n argocd -f

Summary

Key takeaways:

• ✅ Use clear directory structure
• ✅ One ApplicationGenerator per environment
• ✅ One app per file
• ✅ Configure appropriate sync policies
• ✅ Never commit plaintext secrets
• ✅ Monitor and alert on sync status
• ✅ Validate in CI before merge
• ✅ Git is your source of truth

Following these practices ensures reliable, secure, and scalable GitOps with Nyl and ArgoCD.

Migration from Python

This guide helps you migrate from the Python version of nyl to the Rust version.

Compatibility Status

Phase 1: ✅ Compatible

The Rust version in Phase 1 is compatible with existing Python nyl projects for:

• Configuration loading (YAML/JSON)
• Project structure
• Component creation
• Configuration validation

Phase 2+: 🚧 In Progress

Full compatibility for template rendering, Helm integration, and Kubernetes operations will be available in later phases.

Installation

Uninstall Python nyl (Optional)

pip uninstall nyl

Install Rust nyl

cd nyl-rs
cargo install --path .

Or use the pre-built binary from releases.

Configuration

No Changes Needed

Your existing nyl-project.yaml files work as-is:

settings:
  generate_applysets: false
  on_lookup_failure: Error
  components_path: components
  search_path:
    - .
    - lib

TOML Support

TOML support (nyl-project.toml) is planned for Phase 2 or later. For now, use YAML or JSON.

Unknown Fields

The Rust version will warn about unknown fields in the configuration but will not fail. This provides forward compatibility.

Commands

Phase 1 Commands

Phase 2+ Commands

Validation

Run validation to ensure compatibility:

nyl validate

If validation passes, your project is compatible with nyl-rs.

Behavior Differences

Path Resolution

Both versions resolve relative paths the same way:

• Relative to the configuration file’s parent directory
• Absolute paths remain unchanged

Error Messages

Error messages in Rust nyl may be more concise but provide the same information.

Performance

The Rust version is significantly faster:

• Configuration loading: ~5-10x faster
• File operations: ~2-3x faster
• Overall: ~5-10x faster (goal)

Migration Checklist

• Install Rust nyl
• Run nyl validate on existing projects
• Test nyl new component for creating new components
• Verify configuration loading works
• Update CI/CD pipelines to use new binary
• Wait for Phase 2+ for full feature parity

Feature Comparison

Phase 1 Features

Phase 2+ Features

Breaking Changes

None in Phase 1

Phase 1 maintains full backward compatibility with Python nyl.

Future Phases

Breaking changes (if any) will be documented when they are introduced. The goal is to maintain compatibility wherever possible.

Getting Help

If you encounter issues during migration:

1. Run nyl validate --strict to identify problems
2. Check the Configuration documentation
3. Review command documentation for syntax changes
4. File an issue on GitHub

Performance Comparison

Benchmarks show significant performance improvements:

Binary size: 2.0MB (Rust) vs ~50MB (Python with dependencies)

Rollback Plan

If you need to rollback to Python nyl:

# Uninstall Rust nyl
cargo uninstall nyl

# Reinstall Python nyl
pip install nyl

Your project files remain unchanged and will work with either version.

Resources

Nyl provides Kubernetes-style custom resources for declarative configuration and deployment.

Resource Types

Core Resources

• NylRelease: Defines release metadata (name, namespace) for deployments
• HelmChart: Declarative Helm chart deployment with templating support

ArgoCD Resources

• ApplicationGenerator: Automatically generates ArgoCD Applications from NylRelease files

Resource Format

All Nyl resources follow Kubernetes resource conventions:

apiVersion: <api-version>
kind: <resource-kind>
metadata:
  name: <name>
  namespace: <namespace>  # Optional
spec:
  # Resource-specific fields

API Versions

• nyl.niklasrosenstein.github.com/v1: Core Nyl resources (NylRelease, HelmChart)
• argocd.nyl.niklasrosenstein.github.com/v1: ArgoCD integration resources (ApplicationGenerator)

Processing Behavior

Regular Kubernetes Resources

Regular Kubernetes resources (ConfigMap, Deployment, etc.) are passed through unchanged during nyl render.

Nyl Resources

Nyl resources are processed based on their kind:

• NylRelease: Extracted and removed from output (provides metadata only)
• HelmChart: Rendered using Helm templating, replaced with rendered manifests
• ApplicationGenerator: Processed to generate ArgoCD Applications, removed from output

Multi-Document Files

Nyl supports YAML multi-document files with --- separators:

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: myapp
  namespace: default
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
data:
  key: value
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-svc
spec:
  ports:
    - port: 80

Processing:

1. NylRelease is extracted (provides name and namespace)
2. ConfigMap and Service are output as-is

See Also

• Configuration
• Commands

NylRelease

The NylRelease resource specifies release metadata (name and namespace) for a deployment. It’s an optional resource that provides context for rendering manifests.

Resource Definition

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: string        # Release name
  namespace: string   # Target namespace
spec: {}              # Reserved for future use

Field Reference

metadata.name

• Type: string (required)
• Description: The release name
• Usage: Used as the Helm release name when rendering HelmCharts

metadata.namespace

• Type: string (required)
• Description: The target namespace for this release
• Usage: Resources are deployed to this namespace

spec

• Type: object (optional)
• Description: Reserved for future metadata like labels, annotations
• Current: Empty object ({})

Behavior

During Rendering

When nyl render encounters a NylRelease:

1. The resource is extracted from the file
2. Metadata is used for rendering other resources in the file
3. The NylRelease itself is not included in the output

With ApplicationGenerator

When ApplicationGenerator scans files:

1. Files with a NylRelease are discovered
2. An ArgoCD Application is generated per NylRelease
3. Application metadata comes from the NylRelease

Singleton Constraint

Only one NylRelease is allowed per file. Multiple NylReleases in the same file will cause an error:

Error: Multiple NylRelease resources found in file

Examples

Minimal NylRelease

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: myapp
  namespace: default

With Other Resources

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: myapp
  namespace: production
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: myapp-config
  namespace: production
data:
  environment: production
  version: "1.0.0"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
        - name: myapp
          image: myapp:1.0.0

With HelmChart

apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: nginx
  namespace: web
---
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: HelmChart
metadata:
  name: nginx
spec:
  chart:
    repository: https://charts.bitnami.com/bitnami
    name: nginx
    version: "15.4.4"
  release:
    name: nginx      # Uses NylRelease.metadata.name if not specified
    namespace: web   # Uses NylRelease.metadata.namespace if not specified
  values:
    replicaCount: 2

Use Cases

ArgoCD Integration

NylRelease is primarily used for ArgoCD integration:

# app.yaml in Git repository
apiVersion: nyl.niklasrosenstein.github.com/v1
kind: NylRelease
metadata:
  name: myapp
  namespace: production
---
# ... other resources ...

ArgoCD Application:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  source:
    plugin:
      name: nyl
      env:
        - name: NYL_RELEASE_NAME
          value: myapp          # From NylRelease.metadata.name
        - name: NYL_RELEASE_NAMESPACE
          value: production     # From NylRelease.metadata.namespace

ApplicationGenerator Discovery

ApplicationGenerator scans for NylRelease resources:

# apps.yaml
apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
spec:
  source:
    path: clusters/default  # Scans this directory

For each file in clusters/default/ with a NylRelease, an Application is generated.

Metadata Propagation

NylRelease metadata can be used in templates (future enhancement):

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .release.name }}-config
  namespace: {{ .release.namespace }}
data:
  release_name: {{ .release.name }}

Validation

Required Fields

• metadata.name must be specified
• metadata.namespace must be specified

Missing fields cause validation errors:

Error: Invalid NylRelease resource: missing field `metadata.name`

Valid Names

Names must follow Kubernetes naming conventions:

• Lowercase alphanumeric characters, -, .
• Start and end with alphanumeric
• Max 253 characters

When Not to Use

You don’t need NylRelease if:

• Deploying plain Kubernetes manifests without Helm
• Not using ArgoCD integration
• Rendering manifests locally without release context

In these cases, just use regular Kubernetes resources directly.

See Also

• ApplicationGenerator
• ArgoCD Bootstrapping
• HelmChart Resource (future documentation)

ApplicationGenerator Resource Reference

For detailed ApplicationGenerator documentation, see:

ArgoCD ApplicationGenerator Guide

Quick Reference

apiVersion: argocd.nyl.niklasrosenstein.github.com/v1
kind: ApplicationGenerator
metadata:
  name: cluster-apps
  namespace: argocd
spec:
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  source:
    repoURL: https://github.com/org/repo.git
    targetRevision: HEAD
    path: clusters/default
  project: default
  syncPolicy:
    automated:
      prune: true
      selfHeal: true

The ApplicationGenerator resource enables automatic generation of ArgoCD Applications from NylRelease files in a directory.

See the full guide for complete field reference, examples, and usage patterns.