c++ - How to make a threaded network server in Qt?

Question

I'm working on a threaded telnet server (one thread per connection), and can't figure out how to get rid of valgrind errors. I've narrowed the problem down to WHERE I delete the tcpsocket.

I create the QTcpSocket in the run() method of the QThread:

void TelnetConnection::run()
{
    tcpSocketPtr = new QTcpSocket();  
    if (!tcpSocketPtr->setSocketDescriptor(socketDescriptor)) {
        emit error(tcpSocketPtr->error());
        return;
    }
    ....
}

When my app wants to disconnect the client I call:

void TelnetConnection::disconnectClient()
{
    tcpSocketPtr->disconnectFromHost();
}

and the slot which is called upon socket disconnect is: void TelnetConnection::clientDisconnected()

{
    tcpSocketPtr->deleteLater();  
    TelnetConnection::s_clientCount--;
    QThread::quit();  // Exit ths event loop for this thread
}

So, I've tried 1. to delete the QTcpSocket in the clientDisconnected slot but that causes read/write instability. (occasional crash) 2. to deletelater in the clientDisconnected slot but that causes memory read/write errors 3. to delete after the thread's exec loop but that still causes memory read/write errors 4. to deletelater after the thread's exec loop - and all errors are gone.

From what I read, deletelater if called AFTER the exec loop has terminated, will run when the thread is deleted. So while this works, I don't think this is the RIGHT way to to it.

I tried creating the QTcpSocket with "this" as the parent but then my signal connects failed because of parent vs this mismatch errors. (Which would allow the QTcpSocket to be deleted upon thread destruction).

What is the RIGHT way to fix this?

Answer 1

Your problems stem almost entirely from reimplementing QThread. Don't do it. Put all of your functionality into a QObject, and then move it to a bare QThread using moveToThread(). If you only access your object from outside via signal slot connections, you'll be done right then and there.

First of all, I'll always refer to some instance of your TelnetConnection as telnetThread. This is just to make it obvious what thread am I talking about.

The errors in the code you've shown so far, are:

1. You're calling emit error(tcpSocketPtr->error()) from within the run() method. It's called from telnetThread, a different thread than the QObject the signal lives in: it lives in telnetThread->thread().

   The run() method is executing within the telnetThread thread. But the signal's implementation, generated by moc, is expected to be called in whatever thread you instantiated QThread - namely telnetThread->thread(), and that thread can never be equal to the one where run() executes. Basically, somewhat confusingly, the following invariant holds:

   QThread * telnetThread ...
   Q_ASSERT(telnetThread != telnetThread->thread());
   

2. You're calling methods on tcpSocketPtr, living in telnetThread, from slots that execute in another thread. The following holds:

   Q_ASSERT(tcpSocketPtr->thread() == telnetThread);
   

   All of the slots declared on your telnetThread are executing in a different thread from telnetThread itself! So, the body of disconnectClient executes in, say, the GUI thread, but it calls methods directly on tcpSocketPtr.

The following is one way of doing it. It listens on port 8023. ^D ends the connection. Receiving an uppercase Q followed by Enter/Return will cleanly shut down the server.

Introduction

Note: This example has been refactored, and the last server is now properly deleted.

Some care is paid to ensuring that things are wrapped up cleanly. Note that it's OK to simply quit() the running QCoreApplication, the wrap-up will happen automatically. So, all the objects are eventually destructed and freed, and nothing should be crashing. The thread and server emit diagnostic messages to the console from their destructor. This way it's apparent that things do get deleted.

The code supports both Qt 4 and Qt 5.

StoppingThread

Adds a missing behavior to QThread. Normally, when you destruct a running thread, you get a warning message and crash/undefined behavior. This class, upon destruction, tells the thread's event loop to quit and waits for the thread to actually finish. It's used just like QThread would be, except that it won't do silly things upon destruction.

ThreadedQObjectDeleter

Deletes a given QObject when its thread is destroyed. Useful when a thread logically owns its objects. This logical ownership is not a parent-child ownership, because the thread and the logically owned object live in different threads (!).

The constructor is private, and a factory method is provided. This is to enforce creation of the deleter on the free store (a.k.a. the heap). It'd be likely an error to make the deleter on the stack, so this pattern uses the compiler to prevent it from happening.

The object must not have been moved to the specified thread yet, otherwise the construction of the deleter would be subject to race conditions - the object could have already deleted itself within the thread. This precondition is asserted.

ServerFactory

Produces a new server instance when its newConnection slot is invoked. The constructor is passed a QMetaObject of the client QObject to create. Thus this class can construct "any" desired QObject without needing to use templates. There is only one requirement on the object that it creates:

It must have a Q_INVOKABLE constructor taking a QTcpSocket* as the first argument, and QObject *parent as the second argument. The objects it produces are created with parent set to nullptr.

The socket's ownership is transferred to the server.

ThreadedServerFactory

Creates a dedicated StoppingThread for each server made, and moves the server to this thread. Otherwise behaves like ServerFactory. The threads are owned by the factory and are properly disposed of when the factory is destructed.

A server's termination quits the thread's event loop and thus finishes the thread. The finished threads are deleted. Threads that are destructed prior to termination of a server will delete the now-dangling server.

TelnetServer

Implements a trivial telnet server. The interface consists of an invokable constructor. The constructor takes the socket to be used and connects the socket to the internal slots. The functionality is very simple, and the class only reacts to readyRead and disconnected signals from the socket. Upon disconnection, it deletes itself.

This is not really a telnet server, since the telnet protocol is not so trivial. It so happens that telnet clients will work with such dumb servers.

main()

The main function creates a server, a server factory, and connects them together. Then it tells the server to listen for connections on any address, port 8023, and starts the main thread's event loop. The listening server and the factory live in the main thread, but all the servers live in their own threads, as you can easily see when looking at the welcome message. An arbitrary number of servers is supported.

#include <QCoreApplication>
#include <QThread>
#include <QTcpServer>
#include <QTcpSocket>
#include <QAbstractEventDispatcher>
#include <QPointer>

#if QT_VERSION < QT_VERSION_CHECK(5,0,0)
#define Q_DECL_OVERRIDE override
#endif

// A QThread that quits its event loop upon destruction,
// and waits for the loop to finish.
class StoppingThread : public QThread {
    Q_OBJECT
public:
    StoppingThread(QObject * parent = 0) : QThread(parent) {}
    ~StoppingThread() { quit(); wait(); qDebug() << this; }
};

// Deletes an object living in a thread upon thread's termination.
class ThreadedQObjectDeleter : public QObject {
    Q_OBJECT
    QPointer<QObject> m_object;
    ThreadedQObjectDeleter(QObject * object, QThread * thread) :
        QObject(thread), m_object(object) {}
    ~ThreadedQObjectDeleter() {
        if (m_object && m_object->thread() == 0) {
            delete m_object;
        }
    }
public:
    static void addDeleter(QObject * object, QThread * thread) {
        // The object must not be in the thread yet, otherwise we'd have
        // a race condition.
        Q_ASSERT(thread != object->thread());
        new ThreadedQObjectDeleter(object, thread);
    }
};

// Creates servers whenever the listening server gets a new connection
class ServerFactory : public QObject {
    Q_OBJECT
    QMetaObject m_server;
public:
    ServerFactory(const QMetaObject & client, QObject * parent = 0) :
        QObject(parent), m_server(client) {}
    Q_SLOT void newConnection() {
        QTcpServer * listeningServer = qobject_cast<QTcpServer*>(sender());
        if (!listeningServer) return;
        QTcpSocket * socket = listeningServer->nextPendingConnection();
        if (!socket) return;
        makeServerFor(socket);
    }
protected:
    virtual QObject * makeServerFor(QTcpSocket * socket) {
        QObject * server = m_server.newInstance(Q_ARG(QTcpSocket*, socket), Q_ARG(QObject*, 0));
        socket->setParent(server);
        return server;
    }
};

// A server factory that makes servers in individual threads.
// The threads automatically delete itselves upon finishing.
// Destructing the thread also deletes the server.
class ThreadedServerFactory : public ServerFactory {
    Q_OBJECT
public:
    ThreadedServerFactory(const QMetaObject & client, QObject * parent = 0) :
        ServerFactory(client, parent) {}
protected:
    QObject * makeServerFor(QTcpSocket * socket) Q_DECL_OVERRIDE {
        QObject * server = ServerFactory::makeServerFor(socket);
        QThread * thread = new StoppingThread(this);
        connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
        connect(server, SIGNAL(destroyed()), thread, SLOT(quit()));
        ThreadedQObjectDeleter::addDeleter(server, thread);
        server->moveToThread(thread);
        thread->start();
        return server;
    }
};

// A telnet server with following functionality:
// 1. It echoes everything it receives,
// 2. It shows a smiley face upon receiving CR,
// 3. It quits the server upon ^C
// 4. It disconnects upon receiving 'Q'
class TelnetServer : public QObject {
    Q_OBJECT
    QTcpSocket * m_socket;
    bool m_firstInput;
    Q_SLOT void readyRead() {
        const QByteArray data = m_socket->readAll();
        if (m_firstInput) {
            QTextStream out(m_socket);
            out << "Welcome from thread " << thread() << endl;
            m_firstInput = false;
        }
        for (int i = 0; i < data.length(); ++ i) {
            char c = data[i];
            if (c == '04') /* ^D */ { m_socket->close(); break; }
            if (c == 'Q') { QCoreApplication::exit(0); break; }
            m_socket->putChar(c);
            if (c == '
') m_socket->write("
:)", 4);
        }
        m_socket->flush();
    }
public:
    Q_INVOKABLE TelnetServer(QTcpSocket * socket, QObject * parent = 0) :
        QObject(parent), m_socket(socket), m_firstInput(true)
    {
        connect(m_socket, SIGNAL(readyRead()), SLOT(readyRead()));
        connect(m_socket, SIGNAL(disconnected()), SLOT(deleteLater()));
    }
    ~TelnetServer() { qDebug() << this; }
};

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QTcpServer server;
    ThreadedServerFactory factory(TelnetServer::staticMetaObject);
    factory.connect(&server, SIGNAL(newConnection()), SLOT(newConnection()));
    server.listen(QHostAddress::Any, 8023);
    return a.exec();
}

#include "main.moc"