Spring Boot Meets SFTP: A Comprehensive Implementation Guide

In today’s interconnected world, securely transferring files between systems is a common requirement. Secure File Transfer Protocol (SFTP) provides a secure way to transfer files over a network. In this tutorial, we will explore how to implement SFTP in a Spring Boot application using Spring Integration (& Spring SFTP adapters). By the end of this article, you will have working Spring Boot application using capable of uploading downloading and listing files on SFTP server.

What is SFTP?

SSH file transfer protocol (also known as secure file transfer protocol or SFTP) is a network protocol that provides file access, file transfer, and file management over any reliable data streams. In simpler terms, SFTP is a way to securely share files between computers over the internet. Imagine it as a super-safe courier service for digital files.

FTP and FTPS are other similar protocols that are used for file transfer.

Implementation

First, we will have to configure SFTP session factory, with the required parameters like host, port, username, password or private key with passphrase.

DefaultSftpSessionFactory – It helps to setup the basic configuration like host, port, username, password, private key, etc. of the SFTP session. Each time an adapter requests a session, a new SFTP session is created. Internally, spring integration uses Apache MINA SSHD library for SFTP capabilities.

CachingSessionFactory – It is used to cache SFTP sessions to improve performance by reusing existing sessions instead of creating new ones for each request. It maintains a cache of active sessions.

    @Bean
    public SessionFactory<SftpClient.DirEntry> sessionFactory() {
        DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
        factory.setHost("ec2.testhost.com");
        factory.setPort(22);
        factory.setUser("ec2-user");
        factory.setAllowUnknownKeys(true);

        Resource res = new FileSystemResource("/path-to-private-key-file")

        factory.setPrivateKey(res);
        factory.setPrivateKeyPassphrase(null);

        return new CachingSessionFactory<>(factory);
    }

Next, we will have to setup the inbound and outbound channel adapters to handle the file transfer operations. In the example shown here, we will use Java DSL for Spring Integration to configure the adapters. Java DSL for Spring Integration is essentially a façade for Spring Integration. The DSL provides a simple way to configure Spring Integration Message flows in your application using the Builder pattern.

Adapter – It is a component that provides connectivity between spring integration framework and external systems or protocols. SFPT adapters enables file transfer operations over SFTP within a Spring integration application.

IntegrationFlow – This is the main integration DSL abstraction. It helps in defining and configuring message flows between various components like channels, transformers, adapters etc.

Inbound channel adapter – Inbound channel adapters are listeners that connects to the remove server and listens for the remote directory events (like new file being created), at which point it initiates file transfer.

Outbound channel adapter – Outbound channel adapter is a MessageHandler that connects to the remote directory and initiates a file transfer for every file it receives as the payload of an incoming Message.

Sftp Outbound Gateway – SFTP outbound gateway provides a set of commands that helps to interact with a remote SFTP server. It supports bi-directional communication, allowing for both sending and receiving files.

Supported commands: -

• ls (list files)

• nlst (list file names)

• get (retrieve a file)

• mget (retrieve multiple files)

• rm (remove file(s))

• mv (move and rename file)

• put (send a file)

• mput (send multiple files)

MessagingGateway – It abstracts the messaging API provided by Spring integration and lets your application interact with simple interface and be unaware of the underlying complex spring integration APIs.

Inbound Configuration

    @Bean
    public IntegrationFlow inboundConfig() {
        return IntegrationFlow.from(Sftp.inboundAdapter(sessionFactory)
                                .preserveTimestamp(false)
                                .remoteDirectory(remoteDir)
                                .filter(new SftpRegexPatternFileListFilter(Pattern.compile("^.*\\.txt$", Pattern.CASE_INSENSITIVE)))
                                .deleteRemoteFiles(true)
                                .localDirectory(new File("/path-to-local-dir"))
                                .localFilter(new AcceptOnceFileListFilter<>())
                                .maxFetchSize(-1),
                        e -> e.id("inboundAdapter")
                                .autoStartup(true)
                                //Schedule to download files from remote dir
                                .poller(Pollers.cron("0 */5 * * * *").maxMessagesPerPoll(-1)))
                .handle(msg -> {
                    try {
                        File file = (File) msg.getPayload();
                    } catch (Exception e) {
                        //log your error
                    }
                }).get();
    }

Outbound Configuration

    @Bean
    public IntegrationFlow outboundConfig() {
        return IntegrationFlow.from("outboundChannel")
                .handle(Sftp.outboundAdapter(sessionFactory, FileExistsMode.REPLACE)
                        .remoteDirectoryExpression("headers['remote-target-dir']")
                        .autoCreateDirectory(false)
                        .fileNameGenerator(new FileNameGenerator() {
                            @Override
                            public String generateFileName(Message<?> message) {
                                return ((File) message.getPayload()).getName();
                            }
                        })
                ).get();
    }

Testing SFTP configuration

To test our SFTP configuration, I will be using an AWS EC2 instance as a remote SFTP server. While using EC2 instance we need to ensure to enable port 22.

In our application, will setup a spring scheduler to automatically run and send files if available to the remote directory, and for inbound adapter in the configuration we have added CRON expression polling configuration, which will copy any new files from the remote directory to the local directory.

    @Scheduled("0 */5 * * * *")
    public void outboundFileTransfer() {
        try {
            File fileDir = new File(srcPath);
            String fileNames[] = fileDir.list(new FilenameFilter() {
                @Override
                public boolean accept(File dir, String name) {
                    return StringUtils.endsWithIgnoreCase(name, ".txt");
                }
            });

            for (String fileName: fileNames) {
                File file = new File(srcPath + "/" +fileName);
                if (file.exists() && file.isFile()) {
                    gateway.sendFile(file, destPath);
                }
            }
        } catch (Exception e) {
            //log your errors
            e.printStackTrace();
        }
    }

Files transferred to outbound files

Files downloaded to local directory from remote directory

Conclusion

In this article we learned how to configure SFTP Adapter in Spring Boot application using Spring Integration. We learned that we need to configure SFTP session using DefaultSftpSessionFactory and to improve performance we cache the created session using CachingSessionFactory.We saw how to configure SFTP inbound and outbound channel adapter with the help of Java DSL for Spring Integration.

You can find the code for the above example @ Github.