Resource leak for WebSockets closed with NOT_CONSISTENT reason

[Kirill Ilyukhin]

Hello,

When Tomcat receives WebSocket text message with invalid UTF-8, it closes
this connection with NOT_CONSISTENT reason. But after that some objects
(WsSession, UpgradeHandler, etc) stay in heap forever. They are referenced
from AbstractProtocol's connections map.

This leak consistently happens with Tomcat 8.5.3 and 8.5.43, both on Mac OS
and Windows, with or without Tomcat native.

I have created a very simple WebSocket Endpoint which does nothing except
logging its events and incoming messages, please see the code below. Also
you will need a WebSocket client which sends broken UTF-8 in text message
right after connecting to the server.

Thank you,
Kirill

-----------------------
package com.example.wstest;

import org.apache.log4j.Logger;
import javax.websocket.*;

public class WSEndpoint extends Endpoint {
    private static final Logger logger = Logger.getLogger(WSEndpoint.class);
    private WSConnection connection;

    @Override
    public void onOpen(Session session, EndpointConfig config) {
        connection = new WSConnection(session);
        logger.info("Opened WebSocket session-" + session.getId());
    }

    @Override
    public void onClose(Session session, CloseReason closeReason) {
        logger.info("Closed WebSocket session-" + session.getId() + ",
reason: " + closeReason.getCloseCode() + " (" +
closeReason.getReasonPhrase() + ")");
        connection.destroy();
        connection = null;
    }

    @Override
    public void onError(Session session, Throwable throwable) {
        logger.info("Error on WebSocket session-" + session.getId(),
throwable);
        connection.destroy();
        connection = null;
    }

    static class WSConnection implements MessageHandler.Whole<String> {
        private final Session session;

        WSConnection(Session session) {
            this.session = session;
            session.addMessageHandler(this);
        }

        public void destroy() {
            session.removeMessageHandler(this);
        }

        @Override
        public void onMessage(String message) {
            logger.info("Session-" + session.getId() + " onMessage(" +
message  +")");
        }
    }
}
-----------------------