audit-go/README.md

## Audit Log API
The audit log api is a library that simplifies sending audit logs to the audit log system.

### Logging API

The audit api provides two ways to log audit events:

- **Direct logging**  
  Logs are directly validated, serialized and sent via solace.  
  It's the responsibility of the sending service to handle errors and to retry in case of an error.
  ```
  Log(
    ctx context.Context,
    event *auditV1.AuditEvent,
    visibility auditV1.Visibility,
    routingIdentifier *RoutingIdentifier,
    objectIdentifier *auditV1.ObjectIdentifier,
  ) error
  ```

- **Transactional safe logging (transactional outbox pattern)**  
  Functionality is split into two steps:
    - Serialization + Validation  
      The `ValidateAndSerialize` method can be called to validate and serialize the event data.
      The serialized data can be stored in an outbox table / collection in
      the database.
      ```
      ValidateAndSerialize(
        event *auditV1.AuditEvent, 
        visibility auditV1.Visibility, 
        routingIdentifier *RoutingIdentifier, 
        objectIdentifier *auditV1.ObjectIdentifier,
      ) (SerializedPayload, error)
      ```
      
    - Sending data to solace  
      The `Send` method can be used to send the previously serialized data.
      It is the responsibility of the sending service to ensure that the
      method is called (repetitive) until no error is reported.
      ```
      Send(
        ctx context.Context, 
        routingIdentifier *RoutingIdentifier, 
        serializedPayload *SerializedPayload,
      ) error
      ```

### Usage
1. Create Solace client / acl rules
   The topic names have to be compliant to the format as specified
   [here](https://async-api.stackit.schwarz/core-platform/asyncapi.html#operation-publish-stackit-platform/t/swz/audit-log/{region}/{version}/{eventSource}/{additionalParts}).
   A solace client has to be provisioned manually with terraform (for 
   [dev](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/00_dev/essolske01/stackit/client_usernames.tf) /
   [qa](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/02_qs/qssolske01/stackit/client_usernames.tf) /
   [prod](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/03_prod/pssolske01/stackit/client_usernames.tf)).  
   The permission to write to the topic needs to be specified in the terraform configuration (for 
   [dev](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/00_dev/essolske01/stackit/acl_profiles.tf) /
   [qa](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/02_qs/qssolske01/stackit/acl_profiles.tf) /
   [prod](https://dev.azure.com/schwarzit/schwarzit.esb/_git/QaaS-Terraform?path=/03_prod/pssolske01/stackit/acl_profiles.tf)).

2. Instantiation of the messaging API  
   The audit log api uses solace as messaging system. The actual implementation for the 
   connection to the messaging system is implemented by specifying an abstraction and providing a
   concrete implementation for AMQP/Solace.
   ```go
   messagingApi, err := NewAmqpMessagingApi(AmqpConfig{URL: "...", User: "", Password: ""})
   if err != nil { ... }
   ```

3. Instantiation of the audit log api  
   There will be a new Audit API in the future as audit logs will be routed dynamically
   to different data sinks in the future.  
   The plan though is to keep the actual methods that (validates, serializes and) sends the log statements as stable 
   as possible only replacing the API instantiation part later.
   The instantiation of the legacy api is shown below:
   ```
   validator, err := protovalidate.New()
   if err != nil { ... }

   auditApi, err := NewLegacyAuditApi(
       messagingApi, 
       LegacyTopicNameConfig{TopicName: topicName}, 
       validator,
   )
   ```

   For each Solace topic data should be sent to, a new object instance of the legacy API is needed.
   The `messagingAPI` object can be shared across the audit api object instances.

4. Logging audit events  
   As described in [Logging API](#logging-api) messages can be validated, serialized and sent
   through the API.
   It is important to retry as long as errors are returned to ensure that the message is really sent.

### Examples
The test code can be taken as an inspiration how to use the API.
- [api_legacy_test.go](./api_legacy_test.go)

### Tests
For users of the API who integrate the library there is an additional implementation available for
testing purpose. The `MockAuditApi` does not require a connection to a messaging system (therefore, 
does not send any data via messaging). It only serializes and validates data.
It can be instantiated as follows:

```
auditApi, err := NewMockAuditApi()
if err != nil { ... }
```

### Supported data types
| Singular-Type | Plural-Type   | Routable to customer | Description          |
|---------------|---------------|----------------------|----------------------|
| system        | system        | no                   | The STACKIT system   | 
| project       | projects      | yes                  | STACKIT project      | 
| organization  | organizations | yes                  | STACKIT organization |
| folder        | folders       | yes                  | STACKIT folder       |

### Customization

#### os.Stdout logging

The audit log api uses `slog` to print messages on `os.Stdout` as it is part
of the standard library (Go >= 1.21).

To configure slog to print json instead of plain text, the default
logger has to be configured before using it.

```go
import "log/slog"

func main() {
    slog.SetDefault(slog.New(slog.NewJSONHandler(os.Stdout, nil)))
}
```

### Development

#### Build
Before the library can be used, modified or if tests should be executed it's required to generate protobuf files
from the schema definition.

##### Generate Protobuf files

```bash
# Create the gen/java directory manually or rerun the buf generate command
mkdir -p gen/java

cd proto
# --include-imports is required for python code generation
buf generate --include-imports
```

##### Build library
```bash
go mod download && go mod tidy && go get ./... && go fmt ./... && go vet ./... && golangci-lint run && go build ./... && go test ./...
```

#### Register buf validation schema in IntelliJ / Goland
The schema files use `buf` protobuf extensions for validation of constraints.

To register the schema in IntelliJ / Goland clone the repo and add the import path:
```bash
git clone https://github.com/bufbuild/protovalidate.git
```

IntelliJ/Goland > Settings > Languages & Frameworks > Protocol Buffers > Import Paths > + (Add Path) > …/protovalidate/proto/protovalidate

#### Testcontainers
To run the tests **Docker** is needed as [Testcontainers](https://testcontainers.com/) 
is used to run integration tests using a solace docker container.

#### Additional API implementations
There's already an implementation draft of the api for the new dynamically routing 
audit log solution. As the implementation of the system has not officially been 
started yet, it's only a draft with integration tests.
The API code is private to not confuse users or loose data until the new system 
is ready to be used.

The code can be found in the [api_routable.go](./api_routable.go) and 
[api_routable_test.go](./api_routable_test.go) files.

### Java and Python
The repo currently contains additional configuration for Java and Python code generation.

For Python there's an example file showcasing the instantiation and validation,
that needs to be copied to the `gen/python` directory after generating the python-files 
from the `.proto` files.
For Python, it's additionally required to install `protovalidate` on the system as 
described [here](https://github.com/bufbuild/protovalidate-python) or by
running `pip3 install -r requirements.txt`.

In the future related code and configurations for both additional languages (Java and 
Python) will be extracted into separate repositories.

### Open Issues
- Check TODOs in the code
- Finalizing messaging schema
  - Check if fields and constraints are correct and compatible with expected use cases
  - Check that routing api specific parameters are independent of AuditEventLog
- Clarify if `client.go` file can be used for licence / legal reasons  
- Extraction of python / java configurations and code  
- Clean up repo (delete main.go, etc. files)