Automatically sending Transfer-Encoding: chunked but I want to send Content-Length

[MacroMelon]

Hello! I'm trying to send a large file back to the client, and want to have a download progress bar so I need to send the Content-Length header. I know the file size before initiating the transfer, so that's fine. However, I read and send the file in chunks (while checking for backpressure) and this causes uWebsockets to automatically send the Transfer-Encoding: chunked header, with which it is forbidden to send the Content-Length header according to HTTP specification.

I know I could send a custom header such as X-Content-Length, but for sake of general compatibilty reasons I would like to avoid that.

Here is my code to send the file, am I missing something / is there another way to do this?

void initSendFile(HttpResponse<true> *clientConnection, std::string filePath) {

    //start sending a file in chunks to the  client (handles backpressure, creates appropriate onwritable callback)
    std::shared_ptr<std::ifstream> in = std::make_shared<std::ifstream>(filePath, std::ios::in | std::ios::binary);
    uintmax_t fileSize = std::filesystem::file_size(std::filesystem::path(filePath));
    std::shared_ptr<uintmax_t> bytesWritten = std::make_shared<uintmax_t>(0);

    //clientConnection->writeHeader("Content-Length", std::to_string(fileSize));
    std::cout << fileSize << " bytes size" << std::endl;

    //see shared_ptr_cyclic_ownership.txt for why this is safe
    std::function<bool(uintmax_t)> trySendTillEnd = [clientConnection, in, fileSize, bytesWritten](uintmax_t a) -> bool{
        bool keepGoing = true;
        while (keepGoing) {
            uintmax_t totalBytesToSend = fileSize - *bytesWritten;
            //std::cout << totalBytesToSend << std::endl;
            if (totalBytesToSend > bigResponseChunkSize) {
                char buffer[bigResponseChunkSize];
                in->read(buffer, bigResponseChunkSize);
                keepGoing = clientConnection->write(std::string(buffer, bigResponseChunkSize));
                *bytesWritten += bigResponseChunkSize;
            }
            else {
                //final write
                char buffer[totalBytesToSend];
                in->read(buffer, totalBytesToSend);
                clientConnection->write(std::string(buffer, totalBytesToSend));
                clientConnection->end();
                in->close();
                std::cout << "end transfer" << std::endl;
                keepGoing = false;
            }
        }
        return true;
    };

    trySendTillEnd(0);  //the zero is just there as a dummy argument, doesn't do anything (it helps because then I can just use one lambda for both :D )

    clientConnection->onWritable(trySendTillEnd);
}

[MacroMelon]

It turns out I was going about this wrong, I re-wrote it to use tryEnd and onWritable instead of write:

void initSendFile(HttpResponse<true> *clientConnection, std::string filePath) {

    //start sending a file in chunks to the client (handles backpressure, creates appropriate onwritable callback)
    std::shared_ptr<std::ifstream> in = std::make_shared<std::ifstream>(filePath, std::ios::in | std::ios::binary);
    uintmax_t fileSize = std::filesystem::file_size(std::filesystem::path(filePath));
    std::shared_ptr<char[]> buffer = std::make_shared<char[]>(bigResponseChunkSize);

    std::cout << fileSize << " bytes size" << std::endl;

    std::function trySendTillEnd = [clientConnection, in, buffer, fileSize](uintmax_t offset) -> bool {
        //std::cout << offset << " offset backpressure triggered" << std::endl;
        uintmax_t chunkPosition = offset % bigResponseChunkSize;
        //Todo- chunkPosition is always zero? do we even need to account for partial chunks? How does tryEnd and onWritable work?
        //std::cout << chunkPosition << " chunk position" << std::endl;
        std::pair<bool, bool> resp = clientConnection->tryEnd(std::string(std::string(buffer.get(), bigResponseChunkSize), chunkPosition), fileSize);
        bool cont = std::get<0>(resp);
        bool finished = std::get<1>(resp);
        while (cont == true && (!finished)) {
            in->read(buffer.get(), bigResponseChunkSize);
            resp = clientConnection->tryEnd(std::string(buffer.get(), bigResponseChunkSize), fileSize);
            cont = std::get<0>(resp);
            finished = std::get<1>(resp);
        }
        return true;
    };

    clientConnection->onWritable(trySendTillEnd);

    //prefill buffer with data
    in->read(buffer.get(), bigResponseChunkSize);
    trySendTillEnd(0);  //start sending

}

This solution (while maybe not perfect) seems to work great, I haven't run into any issues so far (but do take it with a grain of salt, I am far from an expert)

This also reads the file in chunks so you can send over large files without using huge amounts of RAM.