java - bootstrap.yml configuration not processed anymore with Spring Cloud 2020.0

Question

In my Spring Boot project, I defined 4 profiles

1. demo
2. dev
3. test
4. prod

properties in YAML files will be replaced by HashiCorp Vault properties during startup. For this, I use Spring Cloud Vault library. Everything works as expected in Spring Boot 2.3.x

When I try to upgrade the project to Spring Boot 2.4.0 with Spring Cloud Vault 3.0.0-SNAPSHOT version, the properties are not being replaced

bootstrap.yml

spring:
  cloud:
    vault:
      authentication: APPROLE
      app-role:
        role-id: ${role-id}
        secret-id: ${secret-id}
        role: pres-read
        app-role-path: approle
      uri: ${vault-server}
      connection-timeout: 5000
      read-timeout: 15000
      kv:
        enabled: true
        backend: secret
        application-name: app/pres

application.yml

spring:
  config:
    activate:
      on-profile: 'demo'

application-demo.yml

## Server Properties
server:
  port: 8081

spring:
  datasource:
    username: ${pres.db.username}
    password: ${pres.db.password}
    url: ${pres.db.url}

Spring Cloud Vault Library

      <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-vault-config</artifactId>
            <version>3.0.0-SNAPSHOT</version>
        </dependency>

Error Log:

/Users/jaddap2/Library/Java/JavaVirtualMachines/adopt-openj9-11.0.9/Contents/Home/bin/java -agentlib:jdwp=transport=dt_socket,address=127.0.0.1:59487,suspend=y,server=n -XX:TieredStopAtLevel=1 -noverify -Dspring.profiles.active=demo -Dspring.output.ansi.enabled=always -Drole_id=49c8d8fb-1bdd-7f3b-493f-a11e20907a62 -Dsecret_id=a7ee91cc-e259-82ff-2b34-5c333dda300c -Dcom.sun.management.jmxremote -Dspring.jmx.enabled=true -Dspring.liveBeansView.mbeanDomain -Dspring.application.admin.enabled=true -javaagent:/Users/jaddap2/Library/Caches/JetBrains/IntelliJIdea2020.2/captureAgent/debugger-agent.jar -Dfile.encoding=UTF-8 -classpath /Users/jaddap2/IdeaProjects/HashiCorpVault-SpringCloud/target/classes:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-data-jpa/2.4.0/spring-boot-starter-data-jpa-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-aop/2.4.0/spring-boot-starter-aop-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-aop/5.3.1/spring-aop-5.3.1.jar:/Users/jaddap2/.m2/repository/org/aspectj/aspectjweaver/1.9.6/aspectjweaver-1.9.6.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-jdbc/2.4.0/spring-boot-starter-jdbc-2.4.0.jar:/Users/jaddap2/.m2/repository/com/zaxxer/HikariCP/3.4.5/HikariCP-3.4.5.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-jdbc/5.3.1/spring-jdbc-5.3.1.jar:/Users/jaddap2/.m2/repository/jakarta/transaction/jakarta.transaction-api/1.3.3/jakarta.transaction-api-1.3.3.jar:/Users/jaddap2/.m2/repository/jakarta/persistence/jakarta.persistence-api/2.2.3/jakarta.persistence-api-2.2.3.jar:/Users/jaddap2/.m2/repository/org/hibernate/hibernate-core/5.4.23.Final/hibernate-core-5.4.23.Final.jar:/Users/jaddap2/.m2/repository/org/jboss/logging/jboss-logging/3.4.1.Final/jboss-logging-3.4.1.Final.jar:/Users/jaddap2/.m2/repository/org/javassist/javassist/3.24.0-GA/javassist-3.24.0-GA.jar:/Users/jaddap2/.m2/repository/net/bytebuddy/byte-buddy/1.10.18/byte-buddy-1.10.18.jar:/Users/jaddap2/.m2/repository/antlr/antlr/2.7.7/antlr-2.7.7.jar:/Users/jaddap2/.m2/repository/org/jboss/jandex/2.1.3.Final/jandex-2.1.3.Final.jar:/Users/jaddap2/.m2/repository/com/fasterxml/classmate/1.5.1/classmate-1.5.1.jar:/Users/jaddap2/.m2/repository/org/dom4j/dom4j/2.1.3/dom4j-2.1.3.jar:/Users/jaddap2/.m2/repository/org/hibernate/common/hibernate-commons-annotations/5.1.2.Final/hibernate-commons-annotations-5.1.2.Final.jar:/Users/jaddap2/.m2/repository/org/glassfish/jaxb/jaxb-runtime/2.3.3/jaxb-runtime-2.3.3.jar:/Users/jaddap2/.m2/repository/org/glassfish/jaxb/txw2/2.3.3/txw2-2.3.3.jar:/Users/jaddap2/.m2/repository/com/sun/istack/istack-commons-runtime/3.0.11/istack-commons-runtime-3.0.11.jar:/Users/jaddap2/.m2/repository/com/sun/activation/jakarta.activation/1.2.2/jakarta.activation-1.2.2.jar:/Users/jaddap2/.m2/repository/org/springframework/data/spring-data-jpa/2.4.1/spring-data-jpa-2.4.1.jar:/Users/jaddap2/.m2/repository/org/springframework/data/spring-data-commons/2.4.1/spring-data-commons-2.4.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-orm/5.3.1/spring-orm-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-context/5.3.1/spring-context-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-tx/5.3.1/spring-tx-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-beans/5.3.1/spring-beans-5.3.1.jar:/Users/jaddap2/.m2/repository/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-aspects/5.3.1/spring-aspects-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-web/2.4.0/spring-boot-starter-web-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter/2.4.0/spring-boot-starter-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-logging/2.4.0/spring-boot-starter-logging-2.4.0.jar:/Users/jaddap2/.m2/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:/Users/jaddap2/.m2/repository/ch/qos/logback/logback-core/1.2.3/logback-core-1.2.3.jar:/Users/jaddap2/.m2/repository/org/apache/logging/log4j/log4j-to-slf4j/2.13.3/log4j-to-slf4j-2.13.3.jar:/Users/jaddap2/.m2/repository/org/apache/logging/log4j/log4j-api/2.13.3/log4j-api-2.13.3.jar:/Users/jaddap2/.m2/repository/org/slf4j/jul-to-slf4j/1.7.30/jul-to-slf4j-1.7.30.jar:/Users/jaddap2/.m2/repository/jakarta/annotation/jakarta.annotation-api/1.3.5/jakarta.annotation-api-1.3.5.jar:/Users/jaddap2/.m2/repository/org/yaml/snakeyaml/1.27/snakeyaml-1.27.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-json/2.4.0/spring-boot-starter-json-2.4.0.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.11.3/jackson-databind-2.11.3.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.11.3/jackson-annotations-2.11.3.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.11.3/jackson-core-2.11.3.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jdk8/2.11.3/jackson-datatype-jdk8-2.11.3.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/datatype/jackson-datatype-jsr310/2.11.3/jackson-datatype-jsr310-2.11.3.jar:/Users/jaddap2/.m2/repository/com/fasterxml/jackson/module/jackson-module-parameter-names/2.11.3/jackson-module-parameter-names-2.11.3.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-starter-tomcat/2.4.0/spring-boot-starter-tomcat-2.4.0.jar:/Users/jaddap2/.m2/repository/org/apache/tomcat/embed/tomcat-embed-core/9.0.39/tomcat-embed-core-9.0.39.jar:/Users/jaddap2/.m2/repository/org/glassfish/jakarta.el/3.0.3/jakarta.el-3.0.3.jar:/Users/jaddap2/.m2/repository/org/apache/tomcat/embed/tomcat-embed-websocket/9.0.39/tomcat-embed-websocket-9.0.39.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-web/5.3.1/spring-web-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-webmvc/5.3.1/spring-webmvc-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-expression/5.3.1/spring-expression-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/cloud/spring-cloud-starter-vault-config/3.0.0-SNAPSHOT/spring-cloud-starter-vault-config-3.0.0-20201123.200715-420.jar:/Users/jaddap2/.m2/repository/org/apache/httpcomponents/httpclient/4.5.13/httpclient-4.5.13.jar:/Users/jaddap2/.m2/repository/commons-codec/commons-codec/1.15/commons-codec-1.15.jar:/Users/jaddap2/.m2/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:/Users/jaddap2/.m2/repository/org/springframework/cloud/spring-cloud-starter/3.0.0-SNAPSHOT/spring-cloud-starter-3.0.0-20201123.160603-486.jar:/Users/jaddap2/.m2/repository/org/springframework/cloud/spring-cloud-context/3.0.0-SNAPSHOT/spring-cloud-context-3.0.0-20201123.160522-488.jar:/Users/jaddap2/.m2/repository/org/springframework/security/spring-security-crypto/5.4.1/spring-security-crypto-5.4.1.jar:/Users/jaddap2/.m2/repository/org/springframework/cloud/spring-cloud-commons/3.0.0-SNAPSHOT/spring-cloud-commons-3.0.0-20201123.160533-486.jar:/Users/jaddap2/.m2/repository/org/springframework/security/spring-security-rsa/1.0.9.RELEASE/spring-security-rsa-1.0.9.RELEASE.jar:/Users/jaddap2/.m2/repository/org/bouncycastle/bcpkix-jdk15on/1.64/bcpkix-jdk15on-1.64.jar:/Users/jaddap2/.m2/repository/org/bouncycastle/bcprov-jdk15on/1.64/bcprov-jdk15on-1.64.jar:/Users/jaddap2/.m2/repository/org/springframework/cloud/spring-cloud-vault-config/3.0.0-SNAPSHOT/spring-cloud-vault-config-3.0.0-20201123.200557-420.jar:/Users/jaddap2/.m2/repository/org/springframework/vault/spring-vault-core/2.3.0-M1/spring-vault-core-2.3.0-M1.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-devtools/2.4.0/spring-boot-devtools-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot/2.4.0/spring-boot-2.4.0.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-autoconfigure/2.4.0/spring-boot-autoconfigure-2.4.0.jar:/Users/jaddap2/.m2/repository/mysql/mysql-connector-java/8.0.22/mysql-connector-java-8.0.22.jar:/Users/jaddap2/.m2/repository/org/springframework/boot/spring-boot-configuration-processor/2.4.0/spring-boot-configuration-processor-2.4.0.jar:/Users/jaddap2/.m2/repository/org/projectlombok/lombok/1.18.16/lombok-1.18.16.jar:/Users/jaddap2/.m2/repository/jakarta/xml/bind/jakarta.xml.bind-api/2.3.3/jakarta.xml.bind-api-2.3.3.jar:/Users/jaddap2/.m2/repository/jakarta/activation/jakarta.activation-api/1.2.2/jakarta.activation-api-1.2.2.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-core/5.3.1/spring-core-5.3.1.jar:/Users/jaddap2/.m2/repository/org/springframework/spring-jcl/5.3.1/spring-jcl-5.3.1.jar:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar com.pj.vaultdemo.VaultDemoApplication
Connected to the target VM, address: '127.0.0.1:59487', transport: 'socket'
{spring.web.resources.chain.cache=false, spring.web.resources.cache.period=0}

  .   ____          _            __ _ _
 /\ / ___'_ __ _ _(_)_ __  __ _    
( ( )\___ | '_ | '_| | '_ / _` |    
 \/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::                (v2.4.0)

2020-11-24 14:57:53.788  INFO 95795 --- [  restartedMain] com.pj.vaultdemo.VaultDemoApplication    : Starting VaultDemoApplication using Java 11.0.9 on macOsMachine with PID 95795 (/Users/jaddap2/IdeaProjects/HashiCorpVault-SpringCloud/target/classes started by jaddap2 in /Users/jaddap2/IdeaProjects/HashiCorpVault-SpringCloud)
2020-11-24 14:57:53.792  INFO 95795 --- [  restartedMain] com.pj.vaultdemo.VaultDemoApplication    : The following profiles are active: demo
2020-11-24 14:57:53.895  INFO 95795 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : Devtools property defaults active! Set 'spring.devtools.add-properties' to 'false' to disable
2020-11-24 14:57:53.895  INFO 95795 --- [  restartedMain] .e.DevToolsPropertyDefaultsPostProcessor : For additional web related logging consider setting the 'logging.level.web' property to 'DEBUG'
2020-11-24 14:57:54.904  INFO 95795 --- [  restartedM

Answer 1

As pointed put by Nicoll, With Spring Cloud Vault 3.0 and Spring Boot 2.4, the bootstrap context initialization (bootstrap.yml, bootstrap.properties) of property sources was deprecated. This can be fixed in one of the 2 ways

1. Use Spring Boot 2.4.0 Config Data API to import configuration from Vault (Preferred)
2. Legacy Processing: Enable the bootstrap context either by setting the configuration property spring.cloud.bootstrap.enabled=true or by including the dependency

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>

1. Use Spring Boot 2.4.0 Config Data API (Preferred)

Move bootstrap.yml configuration to application.yml and updated file looks like below

application.yml

spring:
  cloud:
    vault:
      authentication: APPROLE
      app-role:
        role-id: ${role-id}
        secret-id: ${secret-id}
        role: pres-read
        app-role-path: approle
      uri: ${vault-server}
      connection-timeout: 5000
      read-timeout: 15000
  config:
    import: vault://secret/app/pres/

And define individual profiles as shown below. Add spring.config.import: vault://secret/app/pres/demo property.

application-demo.yml

## Server Properties
server:
  port: 8081

spring:
  config:
    import: vault://secret/app/pres/demo
  datasource:
    username: ${pres.db.username}
    password: ${pres.db.password}
    url: ${pres.db.url}
    driver-class-name: com.mysql.cj.jdbc.Driver

Repeat the same process for all profiles like dev, test, qc, staging and prod.

2. Legacy Processing:

Add the following dependency if you still want to use bootstrap.yml

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>

to the project. The issue will be resolved.

See Spring Cloud Vault docs for more information