Small hack of the day

Two things:

  1. syntax highlighting is nice
  2. wouldn’t it be nice to copy /paste it

The main problem is many terminals don’t for no especially good reason allow copy/paste of formatted text. There are three tools which help:

  1. xclip -t text/html this eats standard input and allows pasting of it as HTML, so it can include formatting and color and so on. By default, xclip does plain text, so you have to specify that it’s HTML.
  2. aha this takes ANSI formatted text (i.e. the formatting scheme used by terminals and turns it into HTML).
  3. ¬†unbuffer many tools will only write color output to a terminal, not a pipe. Through the magic of PTYs (pseudoterminals) this program fools other programs into thinking they’re running on a terminal when they’re not.

Now all you have to do is pipe them together. So, take a silly C++ program like this:

void silly;

And compile it like this:

unbuffer g++-7 error.cc -std=c++1z  | aha | xclip -t text/html

Then a simple paste into a box which accepts HTML (e.g. email, etc) gives:

error.cc:2:6: error: variable or field 'silly' declared void
 void silly;
      ^~~~~

 

Advertisements

Server Send Events

Server send events are a way of getting data pushed over an http connection to a browser. The JavaScript interface is very simple. Here’s some useful info:

I thought it would be a neat way of debugging some GIS code in C++: all I had to do was write the data to a socket, and have a page that collected it in JavaScript and plugged it into Google maps. It was that simple, sort of, except that it was actually incredibly awkward getting it up and running since web browsers are both finicky and don’t provide much error information.

All you have to do is open a socket, write the right HTTP headers and send the correct data. I eventually ended up sending it in chunked encoding, which means each message is essentially preceded by a length so the browser knows how much data to accept and put together into a message.¬† The alternative is to use Content-Length and have fixed length messages (like the w3schools example), but I couldn’t manage to get my browser (Firefox) to accept more than one message before closing the connection due to an error. No idea why, but the chunked encoding is much more flexible anyway.

Probably the biggest hurdle is that my HTLM page was just a file, but the server send events were from a (local) server. Either way that meant it was cross domain and so Firefox would block the request because of CORS. Turns out the fix was a simple headre tweak but that had me going for a while!

Anyway, here’s the code:

#include <iostream>
#include <cstring>
#include <cerrno>
#include <vector>
#include <iomanip>


#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/ip.h>


using namespace std;

class VeryBasicServer
{
	private:
		int sockfd=-1; 
		int connection_fd{-1};
	
	public:

		VeryBasicServer()
		{
			//Incantation to create a socket 
			//Address famile: AF_INET (that is internet, aka IPv4)
			//Type is a reliable 2 way stream, aka TCP
			//Note this just creates a socket handle, it doesn't do anything yet.
			//The 0 is the socket type which is pretty redundant for SOCK_STREAM (only one option)
			sockfd = socket(AF_INET, SOCK_STREAM, 0);
			if (sockfd < 0)
				throw runtime_error("Error opening socket:"s + strerror(errno));

			
			//Set options at the socket level (a socket has a stack of levels, and 
			//all of them have options). This one allows reuse of the address, so that
			//if the program crashes, we don't have to wait for the OS to reclaim it, before
			//we can use this socket again. Useful for debugging!
			int true_ = 1; 
			setsockopt(sockfd,SOL_SOCKET,SO_REUSEADDR,&true_,sizeof(int));
			
			
			//Binding a socket is what sets the port number for a particular
			//socket.
			sockaddr_in serv_addr = {};
			serv_addr.sin_family = AF_INET;          //internet address family
			serv_addr.sin_addr.s_addr = INADDR_ANY;  //allow connections from any address
			serv_addr.sin_port = htons(6502);        //Still fighting the 80s CPU wars. 6502 > 8080
			if (bind(sockfd, (struct sockaddr *) &serv_addr, sizeof(serv_addr)) < 0) 
				throw runtime_error("Error binding socket:"s + strerror(errno));
			
			//This is necessary for stream sockets (e.g. TCP). When a client_ connects
			//a new socket will be created just for that connection on a high port.
			listen(sockfd,5);
		}
		

		void read_ignore_and_dump_incoming_data()
		{
			vector<char> buf(65536, 0);
			int n = read(connection_fd,buf.data(),buf.size());
			if (n < 0) 
				throw runtime_error("Error reading from socket:"s + strerror(errno));
			cout << "Message reads: ";
			cout.write(buf.data(), n);
		}

		void accept()
		{
			//Listen blocks until a cnnection is made, then hands over the newly created
			//socket for the connection
			connection_fd = ::accept(sockfd, nullptr, nullptr);

			if (connection_fd < 0) 
				throw runtime_error("Error accepting:"s + strerror(errno));
			
			//We can actually ignore the HTTP header just to get up
			//and running. For ServerSendEvents, there's only one accepted
			//MIME type, and uncompressed is always a valid compression option
			//even if the client_ doesn't request it.
			read_ignore_and_dump_incoming_data();


			//Construct a valid working header.
			//
			//Two important things to note. One is the access control. Since
			//this server isn't serving anything except SSEs, the web page
			//which is using them ust come from elsewhere. Unless we allow 
			//connections from places other than the originating server, then
			//the web browser will block the attempt for security.
			//
			//The other point is the chunked encodeing. The browser connecting
			//has to know when we've finished sending an event in the stream
			//of data. Chunked encoding allows us to send data blocks along with a 
			//length so the server knows when a block is finished. The other option
			//is to have a fixed Content-Length instead. I never got it working,
			//but it's much less flexible so I didn't try hard.
			//
			//Note also the \r\n line terminations, not just \n. Part of the HTTP spec.
			write("HTTP/1.1 200 OK\r\n"                          //Standard HTTP header line
			      "Content-Type: text/event-stream\r\n"          //This is the only allowed MIME type for SSE
                  "Transfer-Encoding: chunked\r\n"               //Chunked encoding lets it know when an event is done without knowing packet boundaries.
				  "Access-Control-Allow-Origin: *\r\n"           //Because the HTML comes from a file, not this server, we have to allow access
			      "\r\n");                                       //End of header indicator

		}
		
		//Write data with less than the minimal amount of error checking.
		void write(const string& str)
		{
			int n = ::write(connection_fd, str.data(), str.size());

			if(n < 0)
				throw runtime_error("Error writing:"s + strerror(errno));
		}

		void write_chunk(const string& str)
		{
			//Chunked encoding is the size in hex followed by the data. Note that both
			//the size and data fields are terminated by HTTP line terminations (\r\n)
			ostringstream o;
			o << hex << str.size() << "\r\n" << str << "\r\n";
			write(o.str());
		}

		~VeryBasicServer()
		{
			cerr << "closing\n";
			close(connection_fd);
			close(sockfd);
		}
};


int main()
{
	VeryBasicServer hax;

	hax.accept();
	
	cout << "Press enter to send an event" << endl;
	for(int i=1;;i++)
	{
		if(cin.get() == EOF)
			break;
		
		//Build up an event in server-send-event format. The message consists of 
		//one or more fields of the form:
		//field: value\n
		//Followed by an empty line.
		//
		//The main tag is "data:" which carries the data payload.
		//See https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events/Using_server-sent_events
		//for more info (e.g. different message types and dispatch)
		string sse;
		sse += "data: " + to_string(i) + "\n";  //Send the current number as the data value
		sse += "\n";                            //Empty field ends the message.

		//Send the message data using chunked encoding
		hax.write_chunk(sse);

	}

}

And the corresponding HTML page is:

<!DOCTYPE html>
<html>
<head>  <meta charset="UTF-8"> </head>
<body>

<h1>Getting server updates</h1>
<div id="result"></div>

<script>
if(typeof(EventSource) !== "undefined") {
    var source = new EventSource("http://127.0.0.1:6502");
    source.onmessage = function(event) {
        document.getElementById("result").innerHTML += "Data: " + event.data + "<br>";
    };
    source.onerror = function(event) {
        document.getElementById("result").innerHTML += "Connection failed<br>";
    };
} else {
    document.getElementById("result").innerHTML = "Sorry, your browser does not support server-sent events...";
}
</script>

</body>
</html>

And that’s basically it. It’s a handy and rather silly chunk you can put in a C++ file. It’s a lot of very poor practices and does all the wrong things for scalability, security, generality and good sense, but it’s handy for debugging hacks.

Code is on github.