Spring Splunk Log4j2

Spring Integration with Splunk Http Event Collector (HEC) & Log4j2

blogs/technical/spring-log4j2-splunk/spring-splunk-log4j2.jpg

In this article I will show how to utilise Splunk's Http Event Collector (HEC) feature to index customised log events via a Spring boot application and a log4j2 appender. Splunk provides the common Universal Forwarder utility to push log automatically from servers to index but this technique will make your application capable of logging into Splunk directly without having a properly configured server infrastructure in place. Check out the github project location [Spring Integration with Splunk & Log4j2](https://github.com/devadyuti/integration-repo/tree/master/spring-log4j2-splunk) for further details  
  
## Setting up a Spring Boot Application  
Create a Springboot application called SpringLog4j2SplunkApplication.java 
```
    package net.ddhome.integration.spring.log4j2.splunk;
    
    import org.springframework.boot.SpringApplication;  
    import org.springframework.boot.autoconfigure.SpringBootApplication;  
      
    @SpringBootApplication  
    public class SpringLog4j2SplunkApplication {  
        public static void main(String[] args) {  
            SpringApplication.run(SpringLog4j2SplunkApplication.class, args);  
        }  
    }
```
pom.xml
```
    <?xml version="1.0" encoding="UTF-8"?>  
    <project xmlns="http://maven.apache.org/POM/4.0.0"  
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
        <modelVersion>4.0.0</modelVersion>  
        <parent>  
            <groupId>org.springframework.boot</groupId>  
            <artifactId>spring-boot-starter-parent</artifactId>  
            <version>2.3.1.RELEASE</version>  
        </parent>  
        <groupId>net.ddhome.integration</groupId>  
        <artifactId>spring-log4j2-splunk</artifactId>  
        <version>1.0.0</version>  
        <packaging>jar</packaging>  
        <name>${project.groupId}::${project.artifactId}</name>  
        <description>Spring Log4j and Splunk Integration Project</description>  
      
        <properties>  
            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
            <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  
            <java.version>14</java.version>  
        </properties>    
      
        <dependencies>  
            <dependency>  
                <groupId>org.springframework.boot</groupId>  
                <artifactId>spring-boot-starter</artifactId>  
                <version>2.3.1.RELEASE</version>
            </dependency> 
        </dependencies>  
      
        <build>  
            <plugins>  
                <plugin>  
                    <groupId>org.springframework.boot</groupId>  
                    <artifactId>spring-boot-maven-plugin</artifactId>  
                    <version>2.3.1.RELEASE</version>  
                </plugin>  
            </plugins>  
        </build>      
    </project>
```

## Enabling Http Event Collector in Splunk   
- Login to Splunk    
 - Go to Settings -> Data Inputs -> Http Event Collector    
 - Click on Global Settings    
 - Beside ‘All Tokens’ click on Enable Button  and hit Save    
    
[![Picture 1](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image001.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image001.png)    
      
## Configure Data Entry for Http Event  
- Go to Settings -> Add Data    
- Click Monitor -> Http Event Collector    
- Populate name and optional source name, description and click Next    
    
[![Picture 2](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image003.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image003.png)   
      
- Select Source Type as Structured -> _json    
      
[![Picture 3](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image005.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image005.png)  
    
- Create a new Index. Enter name and click on Save.    
    
[![Picture 4](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image007.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image007.png)      
    
- Select the Index from Available list to Selected and click Review    
    
[![Picture 5](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image009.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image009.png)      
    
- Hit Submit  
- Capture the token value from last page    
  
## Configure Spunk Log Appender & Log4j2
Modify your pom.xml to effect 3 key changes. 
- Include Splunk repository to load Splunk appender dependency
- Override Springboot's default log configuration to use log4j2 instead
- Add log4j2 and Splunk java logging dependency
 
```
    <?xml version="1.0" encoding="UTF-8"?>  
    <project xmlns="http://maven.apache.org/POM/4.0.0"  
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"  
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">  
        <modelVersion>4.0.0</modelVersion>  
        <parent>  
            <groupId>org.springframework.boot</groupId>  
            <artifactId>spring-boot-starter-parent</artifactId>  
            <version>2.3.0.RELEASE</version>  
        </parent>  
        <groupId>net.ddhome.integration</groupId>  
        <artifactId>spring-log4j2-splunk</artifactId>  
        <version>1.0.0</version>  
        <packaging>jar</packaging>  
        <name>${project.groupId}::${project.artifactId}</name>  
        <description>Spring Log4j and Splunk Integration Project</description>  
      
        <properties>  
            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>  
            <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>  
            <java.version>14</java.version>  
        </properties>  
      
        <repositories>  
            <repository>  
                <id>splunk-artifactory</id>  
                <name>Splunk Releases</name>  
                <url>https://splunk.artifactoryonline.com/artifactory/ext-releases-local</url>  
            </repository>  
        </repositories>  
      
        <dependencies>  
            <dependency>  
                <groupId>org.springframework.boot</groupId>  
                <artifactId>spring-boot-starter</artifactId>  
                <version>2.3.1.RELEASE</version>  
                <exclusions>  
                    <exclusion>  
                        <groupId>org.springframework.boot</groupId>  
                        <artifactId>spring-boot-starter-logging</artifactId>  
                    </exclusion>  
                </exclusions>  
            </dependency>  
            <dependency>  
                <groupId>org.springframework.boot</groupId>  
                <artifactId>spring-boot-starter-log4j2</artifactId>  
                <version>2.3.1.RELEASE</version>  
            </dependency>  
            <dependency>  
                <groupId>com.splunk.logging</groupId>  
                <artifactId>splunk-library-javalogging</artifactId>  
                <version>1.8.0</version>  
            </dependency>  
        </dependencies>  
      
        <build>  
            <plugins>  
                <plugin>  
                    <groupId>org.springframework.boot</groupId>  
                    <artifactId>spring-boot-maven-plugin</artifactId>  
                    <version>2.3.1.RELEASE</version>  
                </plugin>  
            </plugins>  
        </build>  
      
    </project>
```
 Add a Spring configuration to scan for components
```
    package net.ddhome.integration.spring.log4j2.splunk;  
      
    import org.springframework.context.annotation.ComponentScan;  
    import org.springframework.context.annotation.Configuration;  
      
    @Configuration  
    @ComponentScan("net.ddhome.integration.spring.log4j2.splunk")  
    public class SpringLog4jSplunkConfig {  
    }
```
And a component class to log one info and one error event
```
    package net.ddhome.integration.spring.log4j2.splunk.service;  
      
    import org.apache.logging.log4j.LogManager;  
    import org.apache.logging.log4j.Logger;  
    import org.springframework.boot.CommandLineRunner;  
    import org.springframework.stereotype.Service;  
      
    import java.util.Date;  
      
    @Service  
    public class SpringLog4j2SplunkService implements CommandLineRunner {  
        private Logger log = LogManager.getLogger(SpringLog4j2SplunkService.class);  
      
        public void run(String[] args) {  
            log.info("This is a test event in splunk from log4j2 " + new Date());  
      
            log.error("This is a test error in splunk from log4j2 " + new Date());  
        }  
    }
```
And finally configure Http Event Collector details into Spunk Log appender in resources/log4j2-spring.xml file
```
    <?xml version="1.0" encoding="UTF-8"?>  
    <Configuration>  
        <Appenders>  
            <Console name="console" target="SYSTEM_OUT">  
                <PatternLayout  
      pattern="%style{%d{ISO8601}} %highlight{%-5level }[%style{%t}{bright,blue}] %style{%C{10}}{bright,yellow}: %msg%n%throwable" />  
            </Console>  
            <SplunkHttp  
      name="splunkhttp"  
      url="https://localhost:8088"  
      token="YOUR_SPLUNK_HEC_TOKEN"  
      host="localhost"  
      index="http_log_event_collector_idx"  
      type="raw"  
      source="http-log-event"  
      sourcetype="_json"  
      messageFormat="text"  
      disableCertificateValidation="true">  
                <PatternLayout pattern="%m" />  
            </SplunkHttp>      
        </Appenders>  
      
        <Loggers>  
              
      <Root level="info">  
                <AppenderRef ref="console" />  
                <AppenderRef ref="splunkhttp" />  
            </Root>  
        </Loggers>  
    </Configuration>
```
That's it. Let's run and validate the result now
 
## Run Spring Boot Application  
```java -jar target/spring-log4j2-splunk-1.0.0.jar```    
    
[![Picture 6](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image0011.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image0011.png)   
    
## Validate Splunk Index from Search & Reporting
- In Search & Reporting search for index="http_log_event_collector_idx" for last 24 hours    
    
[![Picture 7](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image013.png)](https://taleofddh.s3-eu-west-1.amazonaws.com/images/blogs/technical/spring-log4j2-splunk/clip_image013.png)