Enhanced constexpr

In an earlier post I wrote about using constexpr to generate compile time lookup tables. Unfortunately it turns out that that was dependent on a GCC extension. In actual C++14, pretty much nothing in <cmath> is constexpr.

My brother and I have written a proposal, currently P0533R0 to introduce a number of constexpr functions into <cmath>. Making functions constexpr is slightly tricky because some of them may set the global variable errno and/or floating point exception flags (effectively global variables),  and so it needs to be a language solution, not a pure library one. However, the basic arithmetic operators also fall into similar traps, so it’s not that bad.

The initial paper is to constexpr-ify all “simple” functions which don’t have pointer arguments. In this case, simple is defined as “closed over the extended rationals”, which includes the standard arithmetic operators, comparison operators and things like floor() and ceil(). It turns out there are actually rather a lot of these including many I’d never heard of before ploughing through <cmath>.

I think it should be straightforward to implement since GCC and LLVM both already do compile time evaluation of even non-simple functions (e.g. sqrt), the former actually making them constexpr as an extension of C++14, the latter only using them in the optimizer.

 

 

 

libble++: simple Bluetooth Low Energy on Linux.

Here’s the code on github!

I’ve been working on a Bluetooth Low Energy library for Linux. The emphasis is on modern C++ design, and simplicity of use. I started before BlueZ supported BLE in a meaningful manner, so this library uses bits and bobs from BlueZ, some linked from libbluetooth, and some extracted from the source code.

I don’t make use of any of the GATT (that’s the high level interface, which is implemented over the ATT protocol which itself is sent over bluetooth sockets) code within BlueZ, which is all implemented over DBus. I just construct the packets (that’s the code I pulled out of BlueZ) and send them over a bluetooth socket myself.

With the brief introduction out of the way, how about some code :).

Here’s a complete program which scans for BLE devices:

#include <blepp/lescan.h>

int main()
{
   BLEPP::log_level = BLEPP::LogLevels::Info;
   BLEPP::HCIScanner scanner;
   while (1) {
    std::vector<BLEPP::AdvertisingResponse> ads = scanner.get_advertisements();
   }
}

Is that about the shortest scanning example you’ve ever seen? 😎 I cheated very slightly by bumping up the log level. Otherwise it’d sit there scanning and generating no output whatsoever, which would not be all that much use.

I’ve also not commented this because I think it’s reasonably self evident. There are only two active lines, creating a scanner object and getting the scan results. I also bump up the logging level so it shows scan results without having to print stuff from the ads struct. Not that that is hard, but since more or less everything is optional information, it’s mildly tedious.

With that little hook out of the way, the library follows a socket-and-function design, much like XLib if you happen to know that. In other words, there’s a socket which all the functions use and which is created for you. If you have a simple interaction with BLE to write, you can just use the functions and never worry further. If you have a slightly more complex interaction, for example if you don’t want to block, then you can get the socket and select(), poll(), epoll() or use any other of your favourite system calls.

So as a result, there’s no framework. It’s just a library. It won’t steal your main loop or make it hard to integrate with any other I/O system you happen to have kicking around.

Interacting with a bluetooth device beyond scanning is a bit more complicated, because devices with interaction are a bit more complicated. It’s more or less the same model, except it uses callbacks (set via std::functions), because various bits of information can arrive asynchronously.

Here’s a complete example which shows how to connect to a temperature monitor and log the readings it gives. I’ve highlighted the non-trivial lines so you can see how much code and how much boilerplace there is (not much!):

#include <iostream>
#include <iomanip>
#include <blepp/blestatemachine.h>
#include <blepp/float.h>  //BLE uses the obscure IEEE11073 decimal exponent floating point values
#include <unistd.h>
#include <chrono>
using namespace std;
using namespace chrono;
using namespace BLEPP;

int main(int argc, char **argv)
{
	if(argc != 2)
	{	
		cerr << "Please supply address.\n";
		cerr << "Usage:\n";
		cerr << "prog <addres>";
		exit(1);
	}

	log_level = Error;

	//This class does all of the GATT interactions for you.
	//It's a callback based interface, so you need to provide 
	//callbacks to make it do anything useful. Don't worry: this is
	//a library, not a "framework", so it won't steal your main loop.
	//In other examples, I show you how to get and use the file descriptor, so 
	//you will only make calls into BLEGATTStateMachine when there's data
	//to process.
	BLEGATTStateMachine gatt;

	//This function will be called when a push notification arrives from the device.
	//Not much sanity/error checking here, just for clarity.
	//Basically, extract the float and log it along with the time.
	std::function<void(const PDUNotificationOrIndication&)> notify_cb = [&](const PDUNotificationOrIndication& n)
	{
		auto ms_since_epoch = duration_cast<milliseconds>(system_clock::now().time_since_epoch());
		float temp = bluetooth_float_to_IEEE754(n.value().first+1);

		cout << setprecision(15) << ms_since_epoch.count()/1000. << " " << setprecision(5) << temp << endl;
	};
	
	//This is called when a complete scan of the device is done, giving
	//all services and characteristics. This one simply searches for the 
	//standardised "temperature" characteristic (aggressively cheating and not
	//bothering to check if the service is correct) and sets up the device to 
	//send us notifications.
	//
	//This will simply sit there happily connected in blissful ignorance if there's
	//no temperature characteristic.
	std::function<void()> found_services_and_characteristics_cb = [&gatt, &notify_cb](){
		for(auto& service: gatt.primary_services)
			for(auto& characteristic: service.characteristics)
				if(characteristic.uuid == UUID("2a1c"))
				{
					characteristic.cb_notify_or_indicate = notify_cb;
					characteristic.set_notify_and_indicate(true, false);
				}
	};
	
	//This is the simplest way of using a bluetooth device. If you call this 
	//helper function, it will put everything in place to do a complete scan for
	//services and characteristics when you connect. If you want to save a small amount
	//of time on a connect and avoid the complete scan (you are allowed to cache this 
	//information in certain cases), then you can provide your own callbacks.
	gatt.setup_standard_scan(found_services_and_characteristics_cb);

	//I think this one is reasonably clear?
	gatt.cb_disconnected = [](BLEGATTStateMachine::Disconnect d)
	{
		cerr << "Disconnect for reason " << BLEGATTStateMachine::get_disconnect_string(d) << endl;
		exit(1);
	};
	
	//This is how to use the blocking interface. It is very simple. You provide the main 
	//loop and just hammer on the state machine struct. 
	gatt.connect_blocking(argv[1]);
	for(;;)
		gatt.read_and_process_next();

}

OK so there are come caveats.

Mostly, it’s not finished. I wrote the library to support the work I’m currently doing, and so I’ve not needed to do anything like everything that the BLE protocol supports. BLE also has a lot of features I doubt anyone’s seen in the wild. I don’t like implementing code I can’t use and test, so I’ve not got any interface to features I don’t currently use.

It’ll likely grow over time and get more features, but if there’s something you’d like in there, let me know. It might be easy for me to put in, or I can give pointers to help you write the necessary code.

A long overdue rewrite of my Autoconf tutorial

In 2003, I learned Autoconf in order to write a configure script for libCVD. There were not many good resources for those getting started so I wrote a tutorial.  The tutorial was rather dated and contained some bad practices and missed things I should have covered properly. I also made the mistake of documenting just Autoconf on its own without doing a more in depth treatment of how it integrates with Make. In hindsight that was a mistake. Technically Autoconf is entirely independent of Make. In practice you’re almost always going to use them as a pair and without showing them together it’s hard to convey how to get Autoconf to control Make properly.

I also didn’t have any complete, working examples. Now I have 13.

The new version is now hosted on github:

https://github.com/edrosten/autoconf_tutorial

and on my website here as well:

http://www.edwardrosten.com/code/autoconf/

The best of both worlds

I currently run Ubuntu LTS (14.04 at the moment) on my work laptop and Arch on my home laptop. I like the stability of Ubuntu, and very much like that I  can run security updates without breaking anything. However, the old versions of programs are annoying in some cases. I love running Arch because I always have the latest versions. This bit me today since the version of gerbv I was running was too old to support the high resolution drill placements in Eagle 7 and so my gerbers looked bad.

Give me Archbuntu!

First, get arch-bootstrap.sh and run it in (say) /other_OS to make /other_OS/ARCH_x64. You now have a bootstrapped arch installation. You can chroot into it and it looks arch-like but won’t work properly.

The main reason is that none of the special files are there. Essentially you want everything to mirror your main OS, except for the the OS itself. This can be done with bind mounts. Add the following to /etc/fstab and run mount -a

/proc          /other_OS/ARCH_x64/proc          none defaults,bind 0 0
/sys           /other_OS/ARCH_x64/sys           none defaults,bind 0 0
/dev           /other_OS/ARCH_x64/dev           none defaults,bind 0 0
/tmp           /other_OS/ARCH_x64/tmp           none defaults,bind 0 0
/home          /other_OS/ARCH_x64/home          none defaults,bind 0 0
/etc/passwd    /other_OS/ARCH_x64/etc/passwd    none defaults,bind 0 0
/etc/shadow    /other_OS/ARCH_x64/etc/shadow    none defaults,bind 0 0
/etc/group     /other_OS/ARCH_x64/etc/group     none defaults,bind 0 0
/etc/sudoers   /other_OS/ARCH_x64/etc/sudoers   none defaults,bind 0 0
/etc/sudoers.d /other_OS/ARCH_x64/etc/sudoers.d none defaults,bind 0 0

Now you can chroot into /other_OS/ARCH_x64 and run pacman. Note that /tmp is shared partly to avoid duplication, but also because that’s where the X11 socket it. Without it, you’ll need to use xhost to allow chroot’d programs to connect. And the password related files are duplicated to so sudo still works.

While you can sudo, chroot, then su – to yourself then run programs, that’s a little inconvenient. chroot is root only for security reasons, but since the chroot image is legitimate, we can chroot into that safely as anyone. The following code essentially chroots into /other_OS/ARCH_x64, drops root and then executes its arguments.

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>

int main(int argc, char** argv)
{
	int r = chroot("/other_OS/ARCH_x64/");

	if(argc == 1)
		return 0;
	
	if(r == -1)
	{
		fprintf(stderr, "Error chrooting: %s\n", strerror(errno));
		return 1;
	}
	
	r = setuid(getuid());

	if(r == -1)
	{
		fprintf(stderr, "Error dropping root: %s\n", strerror(errno));
		return 1;
	}

	r = setgid(getgid());

	if(r == -1)
	{
		fprintf(stderr, "Error dropping root: %s\n", strerror(errno));
		return 1;
	}

	execvp(argv[1], argv+1);

	fprintf(stderr, "Error: %s\n", strerror(errno));
}

Compiling this as ae (think arch exec) and putting it in the path allows you to do “ae gerbv” instead of “gerbv” to get the Arch version, for example. Note that unlike the commandline version, the chroot function doesn’t alter the CWD, so it all works just fine.

Now I have the best of both worlds!

Compile time lookup tables

There are many good reasons to have lookup tables computed at compile time. In C++ 14 this is now very easy with generalized constexpr. It took me a while thinking about it before I realised that it really was that easy! Here’s an example of one:

#include <iostream>
#include <cmath>

//Trivial reimplementation of std::array with the operator[]
//methods set to constexpr
template<class T, int N>
struct array
{
	T elems[N];

	constexpr T& operator[](size_t i)
	{
		return elems[i];
	}

	constexpr const T& operator[](size_t i) const
	{
		return elems[i];
	}
};

//Function to build the lookup table
constexpr array<float, 10> foo()
{
	array<float, 10> a = {};

	for(int i=0; i < 10; i++)
		a[i] = exp(i);

	return a;
}

constexpr static array<float, 10> lut = foo<float>();

//This simply serves as a class to be instantiated.
template<int I> struct Silly
{
	Silly()
	{
		std::cout << "Hello " << I << "\n";
	}
};

int main()
{
	//Prove they're compile time constants
	Silly<(int)lut[0]> s0;
	Silly<(int)lut[1]> s1;
	Silly<(int)lut[2]> s2;
	Silly<(int)lut[3]> s3;
	Silly<(int)lut[4]> s4;
	Silly<(int)lut[5]> s5;
	Silly<(int)lut[6]> s6;
	Silly<(int)lut[7]> s7;
	Silly<(int)lut[8]> s8;
	Silly<(int)lut[9]> s9;
}

The only minor wrinkle is that std::array does not have its members set to constexpr to a sufficient degree, so it doesn’t work in this context. However, since few features are needed, a minimal reimplementation is pretty straightforward.

Apparently this is legal.

EDIT: I posted this on /r/cpp and there’s lots of good discussion there. I’m now less sure it’s legal, but still not sure either way.

Apparently, it’s legal to take pointers to elements in a C++ structure and do arithmetic on them to get pointers to other elements (caveats apply).


struct foo
{
    float a, b, c;
};

...

foo f;
float* b_ptr = &f.a + 1;

I’m about 99% sure it’s legal, provided that the class is “standard layout” (a superset of POD—there are no restrictions on constructors or destructors). The standard doesn’t seem to contradict this view. Section 9.2.13 of N3690 (the C++14 final complete draft) specifies that

Nonstatic data members of a (non-union) class with the same access control (Clause 11) are allocated so that later members have higher addresses within a class object.

so a, b, c must all be at increasing addresses. The standard does allow arbitrary padding (9.2.19), but the presence of padding can be detected easily enough (for example sizeof(foo) == 3*sizeof(float). Having a static assert for that would in theory limit portability, but I’ve not encountered a platform where structs of a single scalar type aren’t packed.

It also doesn’t break type punning/strict aliasing rules since a, b, c are of the same type.

It’s possible that 5.7.5 says it’s illegal. 5.7.4 says that a pointer to a non array shall be treated as a pointer to an array of length 1, and 5.7.5 says that going too far over the end of an array (more than 1 element over) is undefined.

However, offsetof is well defined and depends on allowing you to move around within a standard layout class using pointer arithmetic on the pointer to the class.

TL;DR: it’s safe.

OK, why?

Well, one of my TooN2 users asked if it was possible to have a Vector with named elements (such as .x, .y, .z) for a 3-vector, but there are many other possible variants too. If it’s legal, then you could replace float my_data[3]; with float a, b, c;

Here’s an excessively simplified version of TooN to demonstrate the principle:


//Array is one data storage class
#include <array>

//This is another data storage class
struct RGB
{
	float f, g, b;

	float* data()
	{
		return &f;
	}
};

//This is an excessively simplified Vector class. It takes 
//the data storage class and provides operator[]. TooN itself has more layers
//and the size as part of the type, but the principle is the same
template<class Base>
struct Vec: public Base
{
	float* operator[](int i)
	{
		return Base::data()[i];
	}
};

//Define Vec's of various lengths
Vec<std::array<float,2> > v2;
Vec<std::array<float,3> > v3;
Vec<std::array<float,4> > v4;

//Define a Vec of length 3 with named elements
Vec<RGB> vRGB;

Here’s the actual code:
https://github.com/edrosten/TooN/commit/32cb582e8e8f526980e2c7355793d23a3a629c9a

You can now do:

#include <TooN/TooN.h>
#include <TooN/named_elements.h>
#include <TooN/named_elements.h>


//Now actually make some statically allocated vectors with name members.
TOON_MAKE_NAMED_ELEMENT_VECTOR(CMYK, c, m, y, k);
TOON_MAKE_NAMED_ELEMENT_VECTOR(RGB, r, g, b);
TOON_MAKE_NAMED_ELEMENT_VECTOR(XYZ, x, y, z);




int main()
{
        CMYK<> v = TooN::makeVector(1, 2, 3, 4);
        RGB<> r = TooN::makeVector(1, 2, 3);

        std::cout << v << std::endl;
        std::cout << " c = " << v.c 
             << " m = " << v.m 
             << " y = " << v.y 
             << " k = " << v.k << endl;

        cout << v * TooN::makeVector(1, 2, 3, 4) << endl;

}

The code relies on a variadic macro (C++11 inherited the C99 preprocessor which has variadic macros) to generate a vector Base class with the correct named elements in. Note that CMYK, RGB and etc are proper TooN vectors, but (much like slices) they don’t use the same base as a straightforward Vector declaration.

EDIT:

The conclusion from the various discussions is that my technique might not be allowed, though I think it is not 100% clear. A modification is to change the underlying by adding a union so that an array aliases the members:


//Array is one data storage class
#include <array>

//This is another data storage class
struct RGB
{
	union
	{
		struct
		{
			float f, g, b;
		};

		float my_data[3];
	};

	float* data()
	{
		return my_data;
	}
};

It appears from the standard that this is definitively not forbidden, though it’s also not explicitly allowed either.

TooN 3.x

The TooN (and on github) library is now getting updates to bring it in line with C++14 (I’m skipping C++11 as an explicit target). While TooN was essentially complete and functional and hasn’t needed anything significant added in ages, I recently got a “bug” report. I say “bug” in scare quotes since it was actually a design tradeoff I made a while back to try to work around C++ not allowing non-const reference to temporaries.

The tradeoff is that some modify-in-place functions (i.e. normalize) take the argument either by reference or by copy. If the overload can’t take a reference, it copies the argument. Since that is usually caused by having a temporary, such as the result of .slice(), it simply copies the slice which still of course refers to the same data.

However, there are other reasons that a reference can’t be taken. For example, if you pass in a const Vector. Instead of failing because you’re violating constness, the library cheerfully copies the vector, normalises the copy (in place) and then destructs it. The code compiles and silently does nothing at all. Naturally I never documented that pitfall.

So, it’s time to move TooN to the modern world. That little problem is solved beautifully by rvalue references. Once the floodgates are opened, all sorts of things need doing, such as deleting the configure based homebrew/typeof/Boost version of what is now decltype.

TooN-2.0 was about half the length of TooN-1 when it reached feature parity (actually by the time it implemented all the features it had a bunch more just as an artefact of the way it was implemented). This was for a number of reasons, but mostly because it was no longer fighting against non-compliant compilers and weak optimizers. With a better language, the code was shorter and clearer. C++14 will allow me to delete a lot more code, so I fully expect 3.0 to be the shortest release of TooN yet.

To Do:

  • Simplify the code: there’s quite a few classes whose sole purpose is to compute the sometimes quite complex return types. Those can all be replaced with auto.
  • Give Matrix and Vector initialiser list constructors.
  • Replace Data and makeVector with modern variadic functions
  • Make modify-in-place functions take r-value references
  • Move assignment of Vector<Dynamic>
  • Move for Matrix<Dynamic>

Done:

  • R-value reference for normalise()
  • move constructors for Vector<Dynamic>
  • Replace hacky stuff with decltype

However, there’s still the problem of expression templates which I don’t know how to fix. That’ll be the subject of a future post.