appinit

package
v0.0.0-...-0569425 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 1, 2025 License: Apache-2.0 Imports: 21 Imported by: 0

README

Application init and plugins

Unlike Java, in Golang only code that is referenced is linked into the binary. That is good for size - but bad for 'modular' architecture.

Caddy is a http server/reverse proxy that also doubles as a modular application framework - and is very good, but things are tightly coupled and few of the choices are perhaps not perfect. This package is not perfect either.

The model is very simple:

  • any module must be declared in order to be linked in the binary.
  • the 'appinit.Register(string, any)' allows an app to pick what to include
  • go plugins, wasm and processes provide out-of-binary modules.

There is no dependency from 'modules' to the init code.

Why: Using init() pattern from Caddy requires that each module has a dependency on some 'application framework'. I tried using the expvar or the http default mux or other 'globals' in go, but it is not very elegant, and even if it worked - main() still needs to import each module.

Configuration

Golang templates are a very powerful mechnism to 'script' both pages but also initialization, since they allow calls via reflection.

Json is also commonly used - yaml is translated to json.

Patterns

Once the modules are linked in the binary and known by a name, initialization happens in few steps.

  • New - creates the object, with some defaults. If a struct is registered - it will be cloned or created. Builders can also be registered.
  • Fields in the struct are set - using json or reflection
  • Init() or Provision() are called - listeners, etc are created.
  • Start() or Run() are called.
  • Stop() when the server is shutting down - lame duck mode.
  • Restart() if the config is reloaded.
  • Close() if the object is no longer used.

For each step, there are different patterns of parameters - similar to go template language. The object may implement interfaces or have public fields.

Registries or 'Resource Stores'

Protobuf, K8S default libraries - and quite a few others - also provide registries of nameed objects, which can be constructed on demand and loaded from serialized form.

Code generation

Given a list of module names and a map of names to 'known types', it is also possible to generate the initialization code.

Documentation

Index

Constants

View Source 
const ROOT = "mesh"

Variables

View Source 
var ErrNotFound = errors.New("not found")
View Source 
var Unmarshallers = map[string]func([]byte, any) error{}

Just read - not full codec.

View Source 
var Yaml2Json func([]byte) ([]byte, error)

Functions

func Append 

func Append(err1 error, err2 error) error

func ErrorToMap 

func ErrorToMap(err error) map[string]any

func Get 

func Get[T any](ctx context.Context, rs *ResourceStore, key string) (*T, error)

Get is a typed getter, wraps rs.Get()

func Load 

func Load(name string) any

Use go build -buildmode=plugin. go-plugin is using grpc or net/rpc+yamux

func NewSlogError 

func NewSlogError(kind string, args ...any) error

NewSlogError returns a structured error - a slog.Record wrapped in an error. It can be treated as a record error and pushed to a LogHandler, or used directly.

func RecordToMap 

func RecordToMap(record *slog.Record) map[string]any

func RegisterAny 

func RegisterAny(name string, newObj_ any)

func RegisterN 

func RegisterN[T any](name string, newObj_ func() *T)

RegisterN registers a 'New' function with no parameters.

func RegisterT 

func RegisterT[T any](name string, t *T)

RegisterT registers a 'template' object that will be copied.

func Value 

func Value[T any](ctx context.Context, key any) *T

Value is a typed helper for getting a resource.

func WaitEnd 

func WaitEnd()

WaitEnd should be the last thing in a main() app - will block, waiting for SIGTERM and handle draining.

This will also handle any extra args - interpreting them as a CLI and running the command, allowing chaining in docker. Init is using a yaml for config and no CLI.

func Yaml2JSON2 

func Yaml2JSON2(bb []byte) ([]byte, error)

Use go Exec yq to translate to json Yaml is mainly used for human edit, like markdown. It is ok at startup to just use yq to convert.

Types

type Closer 

type Closer interface {
	Close() error
}

type Codecs 

type Codecs struct {
	// contains filtered or unexported fields
}

Codecs is a map of encoding types (json, etc) to interfaces for converting to/from []byte, fs.File or transport and objects that operate on the corresponding resource (can access and possibly modify the content).

A Resource associates metadata with the content. A file system may use 

the URL or file path to encode metadata, along with native metadata

(xattr, .dir files, etc).

Knowing the 'kind' and format using elements inside the object is not ideal (K8S makes it work, but with a high cost). The resource URL and file name, as well as 'content-type' are better places to encode this, as well as 'out of band'.

HTTP Accept-Encoding is one way, but also complex and doesn't work for files.

Using file extensions has the 'readability' advantage of being easy to understand but it is forcing a naming style.

As a middle ground, when using HTTP the headers will take priority, and in configs and APIs the type can be explicitly configured.

This package also deals with registering the 'New' or type object.

func AppCodec 

func AppCodec() *Codecs

func NewCodec 

func NewCodec() *Codecs

func (*Codecs) InitObject 

func (appCodec *Codecs) InitObject(ctx context.Context, r any, data []byte) (any, error)

InitObject will return an object instance with the data unmarshalled into it. 'r' is the registered object - which may be an instance or a constructor. It does not provision the object - just unmarshal

func (*Codecs) New 

func (rs *Codecs) New(name string) any

Given a type or name, return a new instance. This is using the 'New' functions registered in the app or global.

Used by 'Get' to create new instances for unmarshallin.

func (*Codecs) ProcessJSON 

func (appCodec *Codecs) ProcessJSON(ctx context.Context, s string) (map[string]any, error)

ProcessJSON will take a json and use the registered types to unmarshall each key.

func (*Codecs) ProcessJSONSpec 

func (appCodec *Codecs) ProcessJSONSpec(ctx context.Context, s []byte) (any, error)

ProcessJSONSpec handles a K8S-like object with a 'spec' field containing data, Kind, and APIVersion fields describing the kind.

func (Codecs) Range 

func (appCodec Codecs) Range() iter.Seq2[string, interface{}]

type Env 

type Env struct {
	// Parsed environment variables.
	Env map[string]string

	// Local writable directory where configs can be cached and
	// generated files can be written.
	WorkDir string

	// Root directory or URL for config files.
	ConfigDir string

	// Hostname and primary FQDN.
	Hostname string
	FQDN     string
}

type Initializer 

type Initializer interface {
	Init(ctx context.Context, resGet func(ctx2 context.Context, name string) any) error
}

Initializer is using an explicit function to get other modules. Alternative to Provisioner.

type ObjInitializer 

type ObjInitializer interface {
	NewInstance(ctx context.Context) (any, error)
}

type ObjectMeta 

type ObjectMeta struct {
	// Base name, unique in namespace
	Name      string `json:"name,omitempty"`
	Namespace string `json:"namespace,omitempty"`

	ResourceVersion string            `json:"resourceVersion,omitempty"`
	Labels          map[string]string `json:"labels,omitempty"`
	Annotations     map[string]string `json:"annotations,omitempty"`
}

ObjectMeta includes metadata about a resource.

This is in addition to Kind, ApiVersion - which in K8S are top level, but also part of the path.

In K8S this is part of each object - but it doesn't have to be from generic API perspective.

From 'filesystem abstraction' perspective, this is 'extended attributes', and can be represented as a separate database.

Things like embeddings, sha, etc are also metadata.

Different storage systems may hold 'metadata' about a resource.

"Desktop" spec defines Type (Application (.desktop), Link to URL, Directory with .directory extension ) and Name. Application has Path and Exec.

Apple uses ._FILEs (AppleDouble) for metadata. AppleSingle combines meta and file, with a binary header listing the sections. Entry IDs are used.

Tracker (gnome) now uses sqlite.

type Provisioner 

type Provisioner interface {
	Provision(ctx context.Context) error
}

Provisioner is called after unmarshal, before start. It allows to set dependencies and do other initialization. Should not start running anything.

ctx may be used to get Values - which can be dynamic.

type RecordError 

type RecordError struct {
	Record slog.Record
}

SlogError is a list of errors, represented as slog.Record objects. It implements the 'error' interface, returning a Json Array in the same format as JsonHandler. It can also be sent to a LongHandler or slog.Logger directly, resulting on each Record getting sent.

func (RecordError) Error 

func (e RecordError) Error() string

func (*RecordError) Log 

func (e *RecordError) Log(logger *slog.Logger)

func (*RecordError) LogHandle 

func (e *RecordError) LogHandle(logger slog.Handler)

func (*RecordError) Unwrap 

func (e *RecordError) Unwrap() error

type RecordList 

type RecordList struct {
	Records []slog.Record
}

func (*RecordList) Handle 

func (e *RecordList) Handle(ctx context.Context, r slog.Record) error

func (*RecordList) Log 

func (e *RecordList) Log(logger *slog.Logger)

func (*RecordList) LogHandle 

func (e *RecordList) LogHandle(logger slog.Handler)

func (*RecordList) Unwrap 

func (e *RecordList) Unwrap() []error

type Resource 

type Resource struct {
	// BaseName of the object - may use FQDN syntax, but no "/".
	//
	// Naming conventions:
	// - NAME.KIND.ENCODING
	//   - ENCODING can be json, yaml or any registered serializer
	//   - kind is any registered type. K8S apiVersion is mangled replacing . and / with _
	// - the parent dir is the namespace.
	// - suffix is the type - matched in the ResourceKind map. Can be a
	//  long qualfied name like in K8S, but for now aliases are registered.
	// - base name is the name of the object.
	// - resources are loaded on-demand. Once closed, the object can be removed
	// from memory.
	// - some kinds of resources are dirs.
	//
	BaseName string `json:"name,omitempty"`

	// This matches K8S style, it represents extended attributes.
	ObjectMeta `json:"meta,omitempty"`

	// Kind is the type of the object, K8S-based - is an extended attribute
	Kind string `json:"kind,omitempty"`

	// Only used with K8S - appended to Kind, with version omitted for mapping
	// to Codec. Used in REST requests to indicate a K8S-style server.
	APIVersion string `json:"apiVersion,omitempty"`

	Path  string
	Query string

	// References map field names in the object to other config objects.
	// After the 'M' field is populated (before Start), the Ref will be used
	// to set the field to the value of the named object.
	//
	// Start should be able to fill in defaults if needed.
	//
	Ref map[string]string `json:"ref,omitempty"`

	// Context is set for resources that are loaded on demand or ephemeral.
	// Can be a http or gRPC context.
	Context context.Context

	// RawMessage is a []byte, for delayed processing in json.
	// The bytes can be shared or passed without decoding the raw object.
	// Multiple objects can be decoded from same bytes, and can be passed to WASM or other processes.
	Spec json.RawMessage `json:"spec,omitempty"`
	// contains filtered or unexported fields
}

Resource holds a chunk of bytes and metadata, and can be used to create a struct (object).

It implements the fs.FileInfo interface and a partial fs.File, with Read() and Seek() not moving the pos.

The name and metadata are used to identify a struct (object) where data can be unmarshalled and used.

This is modeled as a file, but also based on K8S resource model, which encodes the type and metadata in the same object.

func (*Resource) Close 

func (cfg *Resource) Close() error

func (*Resource) Info 

func (cfg *Resource) Info() (fs.FileInfo, error)

func (*Resource) Init 

func (cfg *Resource) Init(ctx context.Context, a *ResourceStore) error

Init will initialize the resource by unmarshalling the 'spec' and calling Provisioner and other optional interfaces.

func (*Resource) IsDir 

func (cfg *Resource) IsDir() bool

func (*Resource) ModTime 

func (cfg *Resource) ModTime() time.Time

func (*Resource) Mode 

func (cfg *Resource) Mode() fs.FileMode

func (*Resource) Name 

func (cfg *Resource) Name() string

Base name of the file

func (*Resource) Provision 

func (cfg *Resource) Provision(ctx context.Context, a *ResourceStore, o any) error

func (*Resource) Read 

func (cfg *Resource) Read(bytes []byte) (int, error)

Stat, Read and Close are the FS interface.

func (*Resource) ReadAt 

func (cfg *Resource) ReadAt(bytes []byte, off int64) (int, error)

ReadAt is an optimized interface, avoids allocating

func (*Resource) Size 

func (cfg *Resource) Size() int64

func (*Resource) Stat 

func (cfg *Resource) Stat() (fs.FileInfo, error)

func (*Resource) Sys 

func (cfg *Resource) Sys() any

func (*Resource) Type 

func (cfg *Resource) Type() fs.FileMode

Dir interface

type ResourceDir 

type ResourceDir struct {
	BaseName string
}

func (*ResourceDir) Close 

func (cfg *ResourceDir) Close() error

func (*ResourceDir) IsDir 

func (cfg *ResourceDir) IsDir() bool

func (*ResourceDir) ModTime 

func (cfg *ResourceDir) ModTime() time.Time

func (*ResourceDir) Mode 

func (cfg *ResourceDir) Mode() fs.FileMode

func (*ResourceDir) Name 

func (cfg *ResourceDir) Name() string

func (*ResourceDir) Read 

func (cfg *ResourceDir) Read(bytes []byte) (int, error)

func (*ResourceDir) ReadDir 

func (cfg *ResourceDir) ReadDir(n int) ([]fs.DirEntry, error)

func (*ResourceDir) Size 

func (cfg *ResourceDir) Size() int64

func (*ResourceDir) Stat 

func (cfg *ResourceDir) Stat() (fs.FileInfo, error)

func (*ResourceDir) Sys 

func (cfg *ResourceDir) Sys() any

type ResourceFile 

type ResourceFile struct {
	// The Spec field must be set
	*Resource
	// contains filtered or unexported fields
}

ResourceFile implements the io.File - mainly the cursor for Read().

func (*ResourceFile) Read 

func (cfg *ResourceFile) Read(bytes []byte) (int, error)

func (*ResourceFile) Seek 

func (cfg *ResourceFile) Seek(off int64, x int) (int64, error)

type ResourceHolder 

type ResourceHolder interface {
	Resource(ctx context.Context, name string) (any, error)
}

ResourceHolder is like a dir, for named resources. The name should be relative path, with "/" as delimiter.

type ResourceStore 

type ResourceStore struct {
	// Data contains K8S style resources, with a 'kind' encoding the type and a 'spec' encoding the data.
	Data []*Resource `json:"data,omitempty"`

	// LoadedFS contains an in-memory resource filesystem. Key encodes  the type, objects are lazy-loaded.
	// If not found here, the FS will be used.
	LoadedFS map[string]json.RawMessage `json:"fs,omitempty"`

	// Env contains a map of env variables, used instead of os.Getenv
	Env map[string]string `json:"env,omitempty"`

	// BaseDir is a path used as base directory for resources.
	BaseDir string `json:"base,omitempty"`

	Services []string `json:"services"`

	FS fs.FS `json:"-"`

	Logger *slog.Logger
	// contains filtered or unexported fields
}

ResourceStore handles creating and configuring resources (or 'objects').

A 'default' resource store is used for registering New() functions. A store can override the builder.

Resources can be 'data only' (like K8S or proto) or active (like Caddy). Config or runtime data is loaded into the object, and methods may be called.

func AppResourceStore 

func AppResourceStore() *ResourceStore

AppResourceStore returns the default, per app resource store. It is using current dir as root for the configs and for saving. Should NOT be used in tests - use NewResourceStore instead.

func NewResourceStore 

func NewResourceStore() *ResourceStore

NewResourceStore creates a new resource store, using a FS interface for reading configs and a local dir for cache and saving.

func (*ResourceStore) Close 

func (a *ResourceStore) Close() error

func (*ResourceStore) Get 

func (a *ResourceStore) Get(ctx context.Context, name string) (any, error)

Get returns a resource object by name. The name consists of Kind/Name, e.g. "ConfigMap/foo", or only Kind.

If a file or resource with the given name is found - it will be unmarshalled into the object. The result is NOT saved into the 'loaded' objects.

func (*ResourceStore) GetRawJson 

func (a *ResourceStore) GetRawJson(name string) []byte

GetRawJson will do an on-demand resource loading for resources not specified in the config.

func (*ResourceStore) GetResource 

func (a *ResourceStore) GetResource(n string) (*Resource, error)

GetResource returns a resource by name - all files are wrapped in Resource to add metadata.

func (*ResourceStore) Load 

func (rs *ResourceStore) Load(ctx context.Context, base fs.FS, local string) error

func (*ResourceStore) Open 

func (rs *ResourceStore) Open(name string) (fs.File, error)

Implements the fs.FS interface.

func (*ResourceStore) Plugin 

func (a *ResourceStore) Plugin(name string) (any, error)

Plugin loads a plugin compiled with 

`go build -buildmode=plugin -o myplugin.so plugin.go`

It should have a New method.

func (*ResourceStore) Provision 

func (a *ResourceStore) Provision(ctx context.Context) error

For each stored configuration, load the object if a new() function exists.

Named using Caddy conventions (playing with it, as good name as any)

func (*ResourceStore) ReadFile 

func (rs *ResourceStore) ReadFile(name string) ([]byte, error)

func (*ResourceStore) Set 

func (a *ResourceStore) Set(name string, val any) *Resource

Set adds a value to the in-memory resource map. It will be wrapped in a Resource, but doesn't have serialization state.

Values can be injected or used as a registry.

Few pre-defined names are used:

  • json/yaml2json - a function that converts yaml to json
  • json/unmarshaller - a function that unmarshals

func (*ResourceStore) SetResource 

func (a *ResourceStore) SetResource(n string, c *Resource)

func (*ResourceStore) Start 

func (a *ResourceStore) Start() error

If any of the configured objects implements 'Start', call it.

Start implementing Caddy signature.

func (*ResourceStore) String 

func (a *ResourceStore) String() string

type Saver 

type Saver interface {
	// Save will save the resource to an address. If empty, the same address
	// that was used for loading will be used or the default address.
	// This handles both update and create.
	Save(ctx context.Context, addr string) error
}

type Starter 

type Starter interface {
	Start() error
}

type StarterContext 

type StarterContext interface {
	Start(ctx context.Context) error
}

type WithResourceStorer 

type WithResourceStorer interface {
	WithResourceStore(any)
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.