the fail-fast:true property is not triggered if the config map is not found

[jbkervyn]

I think it would be handy (except if there is another way to do this), to make your application fail when no matching configmap can be found in the namespace. I would expect this to be the behavior of fail-fast. However, this only happens when there is an exception while reading the configmap. If the configmap is not found, no exception is triggered.

[wind57]

you're right, we will not fail the app if the configmap or secrets is not found, but that was the behavior for a very long time. I debated this same issue in some PRs, but the response was always the same - this is expected. Over the time, this grew up on me and I agree. If you think about it, lack of some (or incorrect) property read, should result in a different exception in your business logic. I see this in various other places in spring eco-system too, spring projects rarely fail by default on such things.

In general, having such a property here where it would generate a failure could get hairy fast. You can specify more than one configmap for example, and then what should happen? Allow failure of the app for specific ones? Fail if at least one fails? Have a sensible default for all of them? I don't know the correct answer; but having implemented some of these things, it will get complicated :)

btw, you might be the first user reporting and wanting such a feature. If owners of this project think there is a simple way to do this (and if it's worth it), I could implement it.

[wind57]

just dropping a note, if we get to this in the future. may be name such a property strict? And treat it just like we treat profileBasedSources?

[wind57]

I have been thinking about this one over some weekends and this is not trivial at all.

In general, we support two types of searches: by name and by labels. If we think about these two in isolation, then things are not that complicated and we could fail if we can't find what we are looking for. The (big) problem arises when you think about profiles and profiles are a big part of Spring DNA. For example includeProfileSpecificSources is enabled by default, so we will always be looking for profiles. As a matter of fact we add a profile called kubernetes all the time, even if you do not specify one.

As such, if we want to implement this like any other property we have, it could look like : spring.cloud.kubernetes.config.strict and spring.cloud.kubernetes.sources.config. And this is where the first problem arises. Suppose you have a configmap a and you want to say : "if this one is not found - fail", via spring.cloud.kubernetes.config.strict=true. You define such a configmap, but your application will still fail, because a-kubernetes is not present (remember that kubernetes is a profile that we add and includeProfileSpecificSources is on by default). Of course you could set includeProfileSpecificSources=false, but this is not going to solve the problem for users that don't want that.

You could then say, how about:

spring:
  cloud:
    kubernetes:
      config:
         strict: true
      sources:
        - name: a
        - name: a-kubernetes
           strict: false

Well, in such a case, we will have 4 configmaps to read in the end: a, a-kubernetes (a + profile), a-kubernetes (the one defined) and a-kubernetes-kubernetes. We can't tell apart if a-kubernetes-kubernetes is something you defined or something coming from a profile concatenation. We also can't make the assumption that a-kubernetes is actually only a + profile (so no need for a-kubernetes-kubernetes) : that would be guessing from our side, and I am not a fan of this.

We could solve this via:

spring:
  cloud:
    kubernetes:
      config:
         strict: true
      sources:
        - name: a
        - name: a-kubernetes
           strict: false
           include-profile-specific-sources: false

But what if there are 10 profiles total? We would need to repeat:

- name: a-<PROFILE_NAME>
   strict: false
   include-profile-specific-sources: false

10 times in the configuration file. This is ugly, at best and it also couples the profiles way too much to the configuration.

It becomes obvious that spring.cloud.kubernetes.config.strict=true is not really an option.

---

What if we will support only source based strictness? For example:

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: true

The question still remains, how to treat includeProfileSpecificSources? We could introduce one more setting: profile-based-strictness:

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: true
                  profile-based-strictness: false

In such a case, only a configmap presence will be enforced. All other a-<PROFILE_NAME> will be ignored. But then we will have the reverse:

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: true
                  profile-based-strictness: true

So enforce strict for all profiles? Let's say you have a profile dev, do we enforce a-kubernetes and a-dev presence? We could manage this, if users would want a single profile all the time, but what if you are working from local env, but there are two possible databases you can connect to:

@Profile({"dev", "us-west"}) and @Profile({"dev", "us-east"}) are both an option.

It can get even more hairy: what if we treat kubernetes profile separately, since most users don't care about it, but treat other profiles via a specific property.

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: true
                  profile-based-strictness: true
                  kubernetes-profile-strictness: false

In such a case, we would validate that:

• a configmap is present
• a-dev and a-us-west configmaps are present (if dev and us-west profiles are present)
• a-kubernetes is not even tried for presence.

I don't know why but this looks heavily involved, to me.

[wind57]

I was thinking into a slightly different direction here: what if define profiles that you want to have strictness for? For example:

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: false  # this is the default option
                  profiles:
                    - dev
                    - us-west

• This one will try to read (but not fail if these are not present): a, a-dev, a-us-west (notice that a-kubernetes is not here)
• If we change the example above with strict: true, it will validate the presence for a, a-dev, a-us-west.
• If we change the example to:

spring:
   cloud:
      kubernetes:
         config:
            sources:
               - name : a
                  strict: true

it will validate the presence of a, but not enforce a-dev and a-us-west, a-kubernetes (when dev, us-west and kubernetes are active profiles), since you have not provided explicit profiles. These 3 : a-dev, a-us-west and a-kubernetes will still be tried, since nclude-profile-specific-sources is enabled by default.

---

While I am inclined towards this one, the issue comes that we already have a property called include-profile-specific-sources. We can leave it as-is, but profiles will narrow it. So if you say:

spring:
  cloud:
     kubernetes:
        config:
           sources:
              - name : a
                 strict: true
                 include-profile-specific-sources: true
                 profiles: 
                     - dev

and there are two active profiles dev and kubernetes, then a-dev will be enforced and a-kubernetes will not.

I don't mind trying to implement such an approach, but I need @ryanjbaxter thoughts here and yours @jbkervyn too, if you have the time.

Thank you.

[wind57]

note for myself: I started working on this one, and I'll fix one more issue along the way. I've created this fix that makes sure that configmap-a is read before configmap-a-dev for example. If there are two profile based configmaps on the other hand:

• configmap-a
• configmap-a-dev
• configmap-a-prod

we do make any distinction (order wise) between configmap-a-dev and configmap-a-prod. i.e.: you do not know in what order your properties will come from.

What I will be fixing also is being in par with profiles application.yaml according to Spring documentation : last one will win.

[jbkervyn]

I really like your ideas @wind57 . However, what I had in mind was a more simple use-case not going that much into depth with the profiles that you are rightfully pointing out.

What I was thinking is : as long as you do have a configmap mentionned, there should be at least one hit. So if we do take this strict idea that I do like, it would mean : as long as strict = true, the validation would only succeed if a configmap having value either a, a-dev, a-us-west. It would fail if no configmap matching a-* would be found.

Just to ellaborate on the usecase I was initially having when I raised this issue : Simple spring boot app, having no spring.cloud.kubernetes.config.sources property explicitely set. And by the presense of the depencency, would look for a configmap matching the application.name. In this use-case, I thought that it would make sense to have a failing validation when no configmap would be found. That's why I thought having a at-least logic would make sense.

But I must confess that I do not possess a decent knowledge on the topic, so I'm just sharing, but I do value more your expertise over mine.

Thanks a lot for putting so many thoughts into this :)

[wind57]

No worries at all, I liked the idea and I do admit I thought about such a failure of the app when support was added for fail-fast. While I agree with you, think about someone who has a @Profile("dev") (even better if you think they he/she starts the app with an environment property for those profiles) and strict=true. his/her thoughts are going to go to : "if I have this profile, then there should be a way for me to enforce presence of such a source".

imho, what I propose is nothing more then your proposal + some more use-cases, thus more configurations possible, thus more users that are happy.

To your case btw, simply strict=true will suffice indeed and this will be supported. I am almost done with the implementation btw, but it will take a while to properly test this out. thank you for your feedback.

[jbkervyn]

I completely agree with your proposal, I was just thinking, maybe we are making it now a bit too complex/flexible. But I eventually think you're right and that this is the way to go.

[ryanjbaxter]

I am kind of lost from the beginning here, the application should fail to start if fail-fast is set to true

If you want your application to fail the start-up process in such cases, you can set spring.cloud.kubernetes.config.fail-fast=true to make the application start-up fail with an Exception.

[wind57]

fail-fast has only an effect when there is an Exception. To put it simpler:

try {
    // try to read a configmap
catch(Exception any) {
    if (failFast) {
          throw ....
    }
    else {
       // log.error(...)
    }
}

Currently, there is no way to fail for a missing source.