Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

do-not-merge/work-in-progress - Ftr logging link alpha #3181

Open
wants to merge 23 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
12b4562
step 2.2 run
wTylerReid Nov 14, 2024
e0eef2a
run step 2.3
wTylerReid Nov 14, 2024
d02ec92
first attempt at implementing spec and status
wTylerReid Nov 14, 2024
7e9b83a
rerun generate crds
wTylerReid Nov 14, 2024
0cc8ed0
attempt at implementing link reference implementation
wTylerReid Nov 14, 2024
02642f1
fix parent ref
wTylerReid Nov 14, 2024
7bc774f
fix formatting errors
wTylerReid Nov 14, 2024
c99f38e
update reference implementation to something closer to being correct
wTylerReid Nov 14, 2024
8283820
fix field name
wTylerReid Nov 14, 2024
1b29ee3
rerun generate types
wTylerReid Nov 14, 2024
c43ba0f
step 3.1 generate mapper
wTylerReid Nov 14, 2024
7409672
first attempt at adding manual mapper code
wTylerReid Nov 14, 2024
e362e49
4.1 and 4.2 first attempt - didnt implement anything, just ran genera…
wTylerReid Nov 15, 2024
417a5f9
add initial test data directories, copied pasted logging buckets for now
wTylerReid Nov 15, 2024
c1a4531
starting to write mock GCP server, fixing compilation issues
wTylerReid Nov 15, 2024
30d3ac5
working session, continued fixing compilation issues and understand w…
wTylerReid Nov 19, 2024
8a2add4
extract link resource ID from name
wTylerReid Nov 20, 2024
5f5e128
do we care if it works, it compiles :)
wTylerReid Nov 21, 2024
d3a6589
run formatter
wTylerReid Nov 21, 2024
0903713
run make ready-pr
wTylerReid Nov 26, 2024
83ebd48
initial updates for tests, need to move to cloudtop
wTylerReid Nov 26, 2024
49b5b95
working on dynamic tests, resolving some errors
wTylerReid Dec 4, 2024
f22d870
correct harness variable, correct create link request
wTylerReid Dec 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions apis/logging/v1alpha1/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// +kcc:proto=google.logging.v2
package v1alpha1
33 changes: 33 additions & 0 deletions apis/logging/v1alpha1/groupversion_info.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// +kubebuilder:object:generate=true
// +groupName=logging.cnrm.cloud.google.com
package v1alpha1

import (
"k8s.io/apimachinery/pkg/runtime/schema"
"sigs.k8s.io/controller-runtime/pkg/scheme"
)

var (
// GroupVersion is group version used to register these objects
GroupVersion = schema.GroupVersion{Group: "logging.cnrm.cloud.google.com", Version: "v1alpha1"}

// SchemeBuilder is used to add go types to the GroupVersionKind scheme
SchemeBuilder = &scheme.Builder{GroupVersion: GroupVersion}

// AddToScheme adds the types in this group-version to the given scheme.
AddToScheme = SchemeBuilder.AddToScheme
)
193 changes: 193 additions & 0 deletions apis/logging/v1alpha1/link_reference.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package v1alpha1

import (
"context"
"fmt"
"strings"

refsv1beta1 "github.com/GoogleCloudPlatform/k8s-config-connector/apis/refs/v1beta1"
"github.com/GoogleCloudPlatform/k8s-config-connector/pkg/k8s"
apierrors "k8s.io/apimachinery/pkg/api/errors"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
)

var _ refsv1beta1.ExternalNormalizer = &LoggingLinkRef{}

// LoggingLinkRef defines the resource reference to LoggingLink, which "External" field
// holds the GCP identifier for the KRM object.
type LoggingLinkRef struct {
// A reference to an externally managed LoggingLink resource.
// Should be in the format "projects/<projectID>/locations/<location>/buckets/<bucketID>/links/<linkID>".
External string `json:"external,omitempty"`

// The name of a LoggingLink resource.
Name string `json:"name,omitempty"`

// The namespace of a LoggingLink resource.
Namespace string `json:"namespace,omitempty"`

parent *LoggingLinkParent
}

// NormalizedExternal provision the "External" value for other resource that depends on LoggingLink.
// If the "External" is given in the other resource's spec.LoggingLinkRef, the given value will be used.
// Otherwise, the "Name" and "Namespace" will be used to query the actual LoggingLink object from the cluster.
func (r *LoggingLinkRef) NormalizedExternal(ctx context.Context, reader client.Reader, otherNamespace string) (string, error) {
if r.External != "" && r.Name != "" {
return "", fmt.Errorf("cannot specify both name and external on %s reference", LoggingLinkGVK.Kind)
}
// From given External
if r.External != "" {
if _, _, err := parseLoggingLinkExternal(r.External); err != nil {
return "", err
}
return r.External, nil
}

// From the Config Connector object
if r.Namespace == "" {
r.Namespace = otherNamespace
}
key := types.NamespacedName{Name: r.Name, Namespace: r.Namespace}
u := &unstructured.Unstructured{}
u.SetGroupVersionKind(LoggingLinkGVK)
if err := reader.Get(ctx, key, u); err != nil {
if apierrors.IsNotFound(err) {
return "", k8s.NewReferenceNotFoundError(u.GroupVersionKind(), key)
}
return "", fmt.Errorf("reading referenced %s %s: %w", LoggingLinkGVK, key, err)
}
// Get external from status.externalRef. This is the most trustworthy place.
actualExternalRef, _, err := unstructured.NestedString(u.Object, "status", "externalRef")
if err != nil {
return "", fmt.Errorf("reading status.externalRef: %w", err)
}
if actualExternalRef == "" {
return "", k8s.NewReferenceNotReadyError(u.GroupVersionKind(), key)
}
r.External = actualExternalRef
return r.External, nil
}

// New builds a LoggingLinkRef from the Config Connector LoggingLink object.
func NewLoggingLinkRef(ctx context.Context, reader client.Reader, obj *LoggingLink) (*LoggingLinkRef, error) {
id := &LoggingLinkRef{}

// Get Parent
loggingBucketRef, err := refsv1beta1.ResolveLoggingLogBucketRef(ctx, reader, obj, obj.Spec.LoggingLogBucketRef)
if err != nil {
return nil, err
}
projectID := loggingBucketRef.ProjectID
if projectID == "" {
return nil, fmt.Errorf("cannot resolve project")
}
location := loggingBucketRef.Location
bucketID := loggingBucketRef.LoggingLogBucketID
id.parent = &LoggingLinkParent{ProjectID: projectID, Location: location, LogBucket: bucketID}

// Get desired ID
resourceID := valueOf(obj.Spec.ResourceID)
if resourceID == "" {
resourceID = obj.GetName()
}
if resourceID == "" {
return nil, fmt.Errorf("cannot resolve resource ID")
}

// Use approved External
externalRef := valueOf(obj.Status.ExternalRef)
if externalRef == "" {
id.External = asLoggingLinkExternal(id.parent, resourceID)
return id, nil
}

// Validate desired with actual
actualParent, actualResourceID, err := parseLoggingLinkExternal(externalRef)
if err != nil {
return nil, err
}
if actualParent.ProjectID != projectID {
return nil, fmt.Errorf("loggingBucketRef.projectID changed, expect %s, got %s", actualParent.ProjectID, projectID)
}
if actualParent.Location != location {
return nil, fmt.Errorf("loggingBucketRef.location changed, expect %s, got %s", actualParent.Location, location)
}
if actualParent.LogBucket != bucketID {
return nil, fmt.Errorf("loggingBucketRef.LoggingLogBucketID changed, expect %s, got %s", actualParent.Location, bucketID)
}
if actualResourceID != resourceID {
return nil, fmt.Errorf("cannot reset `metadata.name` or `spec.resourceID` to %s, since it has already assigned to %s",
resourceID, actualResourceID)
}
id.External = externalRef
id.parent = &LoggingLinkParent{ProjectID: projectID, Location: location, LogBucket: bucketID}
return id, nil
}

func (r *LoggingLinkRef) Parent() (*LoggingLinkParent, error) {
if r.parent != nil {
return r.parent, nil
}
if r.External != "" {
parent, _, err := parseLoggingLinkExternal(r.External)
if err != nil {
return nil, err
}
return parent, nil
}
return nil, fmt.Errorf("LoggingLinkRef not initialized from `NewLoggingLinkRef` or `NormalizedExternal`")
}

type LoggingLinkParent struct {
ProjectID string
Location string
LogBucket string
}

func (p *LoggingLinkParent) String() string {
return "projects/" + p.ProjectID + "/locations/" + p.Location + "/buckets/" + p.LogBucket
}

func asLoggingLinkExternal(parent *LoggingLinkParent, resourceID string) (external string) {
return parent.String() + "/links/" + resourceID
}

func parseLoggingLinkExternal(external string) (parent *LoggingLinkParent, resourceID string, err error) {
external = strings.TrimPrefix(external, "/")
tokens := strings.Split(external, "/")
if len(tokens) != 8 || tokens[0] != "projects" || tokens[2] != "locations" || tokens[4] != "bucket" || tokens[6] != "link" {
return nil, "", fmt.Errorf("format of LoggingLink external=%q was not known (use projects/<projectId>/locations/<location>/buckets/<bucketID>/links/<linkID>)", external)
}
parent = &LoggingLinkParent{
ProjectID: tokens[1],
Location: tokens[3],
LogBucket: tokens[5],
}
resourceID = tokens[7]
return parent, resourceID, nil
}

func valueOf[T any](t *T) T {
var zeroVal T
if t == nil {
return zeroVal
}
return *t
}
109 changes: 109 additions & 0 deletions apis/logging/v1alpha1/link_types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package v1alpha1

import (
refs "github.com/GoogleCloudPlatform/k8s-config-connector/apis/refs/v1beta1"
"github.com/GoogleCloudPlatform/k8s-config-connector/pkg/apis/k8s/v1alpha1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

var LoggingLinkGVK = GroupVersion.WithKind("LoggingLink")

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

// LoggingLinkSpec defines the desired state of LoggingLink
// +kcc:proto=google.logging.v2.Link
type LoggingLinkSpec struct {
// +kubebuilder:validation:XValidation:rule="self == oldSelf",message="ResourceID field is immutable"
// Immutable.
// The LoggingLink name. If not given, the metadata.name will be used.
ResourceID *string `json:"resourceID,omitempty"`

// Describes this link.
// The maximum length of the description is 8000 characters.
Description *string `json:"description,omitempty"`

// Placeholder description
LoggingLogBucketRef *refs.LoggingLogBucketRef `json:"loggingLogBucketRef,omitempty"`
}

// LoggingLinkStatus defines the config connector machine state of LoggingLink
type LoggingLinkStatus struct {
/* Conditions represent the latest available observations of the
object's current state. */
Conditions []v1alpha1.Condition `json:"conditions,omitempty"`

// ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.
ObservedGeneration *int64 `json:"observedGeneration,omitempty"`

// A unique specifier for the LoggingLink resource in GCP.
ExternalRef *string `json:"externalRef,omitempty"`

// ObservedState is the state of the resource as most recently observed in GCP.
ObservedState *LoggingLinkObservedState `json:"observedState,omitempty"`
}

// LoggingLinkObservedState is the state of the LoggingLink resource as most recently observed in GCP.
type LoggingLinkObservedState struct {
// We need to add a reference here to lifecycle state and Bigquery Dataset here since they are in the proto

// +optional
// +kubebuilder:validation:Format=date-time
CreateTime *string `json:"createTime,omitempty"`

// the lifecycle state might be something more complicated
// this is an ENUM, should be safe to use a string
LifecycleState *string `json:"lifecycleState,omitempty"`

// this field is just a string, but its an object(string)
// https://github.com/googleapis/googleapis/blob/master/google/logging/v2/logging_config.proto#L1063
BigQueryDataset *BigQueryDataset `json:"bigQueryDataset,omitempty"`
}

// +genclient
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// TODO(user): make sure the pluralizaiton below is correct
// +kubebuilder:resource:categories=gcp,shortName=gcplogginglink;gcplogginglinks
// +kubebuilder:subresource:status
// +kubebuilder:metadata:labels="cnrm.cloud.google.com/managed-by-kcc=true";"cnrm.cloud.google.com/system=true"
// +kubebuilder:printcolumn:name="Age",JSONPath=".metadata.creationTimestamp",type="date"
// +kubebuilder:printcolumn:name="Ready",JSONPath=".status.conditions[?(@.type=='Ready')].status",type="string",description="When 'True', the most recent reconcile of the resource succeeded"
// +kubebuilder:printcolumn:name="Status",JSONPath=".status.conditions[?(@.type=='Ready')].reason",type="string",description="The reason for the value in 'Ready'"
// +kubebuilder:printcolumn:name="Status Age",JSONPath=".status.conditions[?(@.type=='Ready')].lastTransitionTime",type="date",description="The last transition time for the value in 'Status'"

// LoggingLink is the Schema for the LoggingLink API
// +k8s:openapi-gen=true
type LoggingLink struct {
metav1.TypeMeta `json:",inline"`
metav1.ObjectMeta `json:"metadata,omitempty"`

// +required
Spec LoggingLinkSpec `json:"spec,omitempty"`
Status LoggingLinkStatus `json:"status,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
// LoggingLinkList contains a list of LoggingLink
type LoggingLinkList struct {
metav1.TypeMeta `json:",inline"`
metav1.ListMeta `json:"metadata,omitempty"`
Items []LoggingLink `json:"items"`
}

func init() {
SchemeBuilder.Register(&LoggingLink{}, &LoggingLinkList{})
}
27 changes: 27 additions & 0 deletions apis/logging/v1alpha1/types.generated.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading