diff --git a/README.md b/README.md index b4c3655..962ea4b 100644 --- a/README.md +++ b/README.md @@ -1,113 +1,18 @@ -## Audit Log API -The audit log api is a library that simplifies sending audit logs to the audit log system. +## audit-go -### Logging API +The audit-go library is the core library for validation and sending of audit events. -The audit api provides two ways to log audit events: +### API Documentation -- **Direct logging** - Logs are directly validated, serialized and sent via solace. - It's the responsibility of the sending service to handle errors and to retry in case of an error. - ``` - Log( - ctx context.Context, - event *auditV1.AuditEvent, - visibility auditV1.Visibility, - routingIdentifier *RoutingIdentifier, - objectIdentifier *auditV1.ObjectIdentifier, - ) error - ``` +The api documentation can be found [here](). -- **Transactional safe logging (transactional outbox pattern)** - Functionality is split into two steps: - - Serialization + Validation - The `ValidateAndSerialize` method can be called to validate and serialize the event data. - The serialized data can be stored in an outbox table / collection in - the database. - ``` - ValidateAndSerialize( - event *auditV1.AuditEvent, - visibility auditV1.Visibility, - routingIdentifier *RoutingIdentifier, - objectIdentifier *auditV1.ObjectIdentifier, - ) (SerializedPayload, error) - ``` - - - Sending data to solace - The `Send` method can be used to send the previously serialized data. - It is the responsibility of the sending service to ensure that the - method is called (repetitive) until no error is reported. - ``` - Send( - ctx context.Context, - routingIdentifier *RoutingIdentifier, - serializedPayload *SerializedPayload, - ) error - ``` +TODO update the link -### Usage -1. Create Solace client / acl rules - The topic names have to be compliant to the format as specified - [here](https://async-api.stackit.schwarz/core-platform/asyncapi.html#operation-publish-stackit-platform/t/swz/audit-log/{region}/{version}/{eventSource}/{additionalParts}). - A solace client has to be provisioned manually with terraform (for - [dev](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/00_dev/essolske01/stackit/client_usernames.tf) / - [qa](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/02_qs/qssolske01/stackit/client_usernames.tf) / - [prod](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/03_prod/pssolske01/stackit/client_usernames.tf)). - The permission to write to the topic needs to be specified in the terraform configuration (for - [dev](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/00_dev/essolske01/stackit/acl_profiles.tf) / - [qa](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/02_qs/qssolske01/stackit/acl_profiles.tf) / - [prod](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/03_prod/pssolske01/stackit/acl_profiles.tf)). +### Supported data types for routing -2. Instantiation of the messaging API - The audit log api uses solace as messaging system. The actual implementation for the - connection to the messaging system is implemented by specifying an abstraction and providing a - concrete implementation for AMQP/Solace. - ```go - messagingApi, err := NewAmqpMessagingApi(AmqpConfig{URL: "...", User: "", Password: ""}) - if err != nil { ... } - ``` +The following data types are currently supported for routing. +There are `SingularType` and `PluralType` representations in the code for them. -3. Instantiation of the audit log api - There will be a new Audit API in the future as audit logs will be routed dynamically - to different data sinks in the future. - The plan though is to keep the actual methods that (validates, serializes and) sends the log statements as stable - as possible only replacing the API instantiation part later. - The instantiation of the legacy api is shown below: - ``` - validator, err := protovalidate.New() - if err != nil { ... } - - auditApi, err := NewLegacyAuditApi( - messagingApi, - LegacyTopicNameConfig{TopicName: topicName}, - validator, - ) - ``` - - For each Solace topic data should be sent to, a new object instance of the legacy API is needed. - The `messagingAPI` object can be shared across the audit api object instances. - -4. Logging audit events - As described in [Logging API](#logging-api) messages can be validated, serialized and sent - through the API. - It is important to retry as long as errors are returned to ensure that the message is really sent. - -### Examples -The test code can be taken as an inspiration how to use the API. -- [api_legacy_test.go](./api_legacy_test.go) - -### Tests -For users of the API who integrate the library there is an additional implementation available for -testing purpose. The `MockAuditApi` does not require a connection to a messaging system (therefore, -does not send any data via messaging). It only serializes and validates data. -It can be instantiated as follows: - -``` -auditApi, err := NewMockAuditApi() -if err != nil { ... } -``` - -### Supported data types | Singular-Type | Plural-Type | Routable to customer | Description | |---------------|---------------|----------------------|----------------------| | system | system | no | The STACKIT system | @@ -115,89 +20,63 @@ if err != nil { ... } | organization | organizations | yes | STACKIT organization | | folder | folders | yes | STACKIT folder | -### Customization +### Additional API implementations -#### os.Stdout logging +There's already an implementation draft of the api for the new dynamically routing +audit log solution. As the implementation of the system has not officially been +started yet, it's only a draft with integration tests. +The API code is private to not confuse users or loose data until the new system +is ready to be used. -The audit log api uses `slog` to print messages on `os.Stdout` as it is part -of the standard library (Go >= 1.21). - -To configure slog to print json instead of plain text, the default -logger has to be configured before using it. - -```go -import "log/slog" - -func main() { - slog.SetDefault(slog.New(slog.NewJSONHandler(os.Stdout, nil))) -} -``` +The code can be found in the [api_routable.go](./api_routable.go) and +[api_routable_test.go](./api_routable_test.go) files. ### Development -#### Build -Before the library can be used, modified or if tests should be executed it's required to generate protobuf files -from the schema definition. +#### Schema Generation -##### Generate Protobuf files +Go structs are generated from Protobuf schema by using [Buf](https://buf.build). +The schema generator also generates code to validate constraints specified +in the schema. -```bash -# Create the gen/java directory manually or rerun the buf generate command -mkdir -p gen/java +It may be required to install required plugins manually by running: -cd proto -# --include-imports is required for python code generation -buf generate --include-imports +```shell +go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.34.2 +go install github.com/envoyproxy/protoc-gen-validate@v1.1.0 ``` -##### Build library +Please check that the version matches the version in the *go.mod* file. + +Then the schema can be generated: + +```bash +cd proto +buf generate +``` + +#### Build + +The library can be built by executing the following commands: + ```bash go mod download && go mod tidy && go get ./... && go fmt ./... && go vet ./... && golangci-lint run && go build ./... && go test ./... ``` +##### Testcontainers + +To run the tests **Docker** is needed as [Testcontainers](https://testcontainers.com/) +is used to run integration tests using a solace docker container. + #### Register buf validation schema in IntelliJ / Goland -The schema files use `buf` protobuf extensions for validation of constraints. + +The schema files use `Buf` protobuf extensions for validation of constraints. To register the schema in IntelliJ / Goland clone the repo and add the import path: + ```bash git clone https://github.com/bufbuild/protovalidate.git ``` -IntelliJ/Goland > Settings > Languages & Frameworks > Protocol Buffers > Import Paths > + (Add Path) > …/protovalidate/proto/protovalidate - -#### Testcontainers -To run the tests **Docker** is needed as [Testcontainers](https://testcontainers.com/) -is used to run integration tests using a solace docker container. - -#### Additional API implementations -There's already an implementation draft of the api for the new dynamically routing -audit log solution. As the implementation of the system has not officially been -started yet, it's only a draft with integration tests. -The API code is private to not confuse users or loose data until the new system -is ready to be used. - -The code can be found in the [api_routable.go](./api_routable.go) and -[api_routable_test.go](./api_routable_test.go) files. - -### Java and Python -The repo currently contains additional configuration for Java and Python code generation. - -For Python there's an example file showcasing the instantiation and validation, -that needs to be copied to the `gen/python` directory after generating the python-files -from the `.proto` files. -For Python, it's additionally required to install `protovalidate` on the system as -described [here](https://github.com/bufbuild/protovalidate-python) or by -running `pip3 install -r requirements.txt`. - -In the future related code and configurations for both additional languages (Java and -Python) will be extracted into separate repositories. - -### Open Issues -- Check TODOs in the code -- Finalizing messaging schema - - Check if fields and constraints are correct and compatible with expected use cases - - Check that routing api specific parameters are independent of AuditEventLog -- Clarify if `client.go` file can be used for licence / legal reasons -- Extraction of python / java configurations and code -- Clean up repo (delete main.go, etc. files) -- Update dependencies +IntelliJ/Goland > Settings > Languages & Frameworks > Protocol Buffers > Import Paths > + +(Add Path) > …/protovalidate/proto/protovalidate