1
2
3
4
5
6
7
8
9
10

template <typename... Args> struct count;
 
template <typename T, typename... Args> struct count<T, Args...> {
        static const int num = count<Args...>::num + 1;
};
 
template <> struct count<> { static const int num = 0; };
 
// let's use the struct
const int numArgs = count<1, 2, 3, 4, 5, 6, 7, 8, 9, 10>; // numArgs will be equal to 10 because there are 10 parameters given to the count struct

Line 1, is pretty much the root of all template metaprogramming trickery. The fundamentals of it all relies on recursive partial specialization, and line 1 just sets up a forward declaration to the count struct, which never actually gets used. There’s some complicated rules to how partial specializations are chosen by a C++ compiler, but the basic rule of thumb is that the most specialized version of the template that applies is what’s chosen. So although our instantiation at line 10 could match our blank forward declaration, it doesn’t. The meat of our tiny template metaprogram gets called instead: the recursive partial specialization at line 3.

And this partial specialization brings us to the key behind using variadic templates in the real-world. Since it’s not possible to do random access on the pack elements, the only choice is to “unroll” the packs recursively. The trick is to define the partial specialization such that the first argument is just a single parameter, and the next is a parameter pack:

template <typename FirstArg, typename... RemainingArgs> struct count<T, Args...>;

When the above struct is instantiated with a parameter pack, the first element gets pulled out of the pack into FirstArg, and the remaining parameters are placed in RemainingArgs. So, if the count struct makes an instantiation of itself with RemainingArgs... as the only instantiation parameter, again the first parameter will get pulled out and become FirstArg, and the rest will get placed in RemainingArgs. Thus, we can now easily unroll any parameter pack!

Let’s go back to the first code listing at line 4. This is where the recursion happens, and we can see exactly what we were talking about above happening. The count struct calls itself with the remaining arguments, and simply keeps track of the number of times the recursion happens by incrementing a counter along the way. When the instantiation finishes count::num will be set to the number of parameters in the parameter pack as of the initial instantiation plus 1. We add one because technically the recursion only counts the number of items in the parameter pack which, due to our partial specialization, is going to be one less than the number of parameters we pass to count (remember the first element gets picked out of the parameter pack).

The next thing we need to do, like in all forms of recursion, is to define the termination condition of the recursion. There are a number of ways to do this, but in this first example, we employ the “empty partial specialization” trick:

template <> struct count<> { static const int num = 0; };

This terminates the recursion not because num is implicitly set to zero, but because this specialization does not call itself. Therefore when the recursion has completely unrolled the parameter pack we hit this empty specialization and since there is no recursive call, nothing else happens. We only bother to define num here so that if someone explicitly instantiates a count struct with no arguments, they can still call count::num and get a value of zero rather than a compilation error.

Alright, so that was fun, now let’s look at something a little more complex, and a lot more useful:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

template <int N, typename... Args> struct elementType;
 
template <int N, typename T, typename... Args>
struct elementType<N, T, Args...> {
        static_assert(N < sizeof...(Args) + 1, "overflow!");
        typedef typename elementType<N - 1, Args...>::type type;
};
 
template <typename T, typename... Args>
struct elementType<0, T, Args...> {
        typedef T type;
};
 
// now let's use our elementType struct
elementType<0, int, char, std::string, double>::type var0 = 1; // var0 is an int
elementType<1, int, char, std::string, double>::type var1 = '2'; // var1 is a char
elementType<2, int, char, std::string, double>::type var2 = ";P"; // var2 is a string
elementType<3, int, char, std::string, double>::type var3 = 3.14159; // var3 is a double

The basic idea behind the elementType struct is that we give the struct an index and a variable number of types, and we can then use the struct’s type definition as a custom data type that maps back to whatever parameter the index value is associated with. The above listing at lines 15 through 18 do exactly that.

What’s so great about that, and why would such a thing be useful? While it may not seem like such an awesome thing to do at first, what we’ve done is found a way to pick elements out of a parameter pack by an index. Essentially we’ve given ourselves a way to allow random access to packs. The technique used is similar to the count struct, but not quite the same. We’re still using recursion, but this time we need to stop the recursion at the right element.

Take a look at line 6 in the above listing. This is where all the magic happens. The elementType struct calls itself using the same variadic unrolling trick that count used, except we decrement N at each step of the way, until N reaches zero, and we hit the termination condition (the partial specialization for when N is zero at lines 9 through 12). This means if we give an index of 5, we’ll get 5 recursions and the typedef that ends up getting defined is the one for the 6th parameter in the pack.

No problem right! So let’s put all this together for something actually useful — a variadic tuple class:

1
2
3
4
5
6

template <typename... Args> 
struct tuple {
        tuple(const Args&... args) {}
};
 
tuple<int, char> t1(1, '2');

The idea is we want to be able to use our tuple class like line 6 shows. This is the example from Part I, and as I previously mentioned, this isn’t so useful. Sure it lets us declare a tuple like we want, but it doesn’t actually store the tuple’s constructor arguments, nor does it give us any way to retrieve them. So how are we going to store the elements?

Like all things variadic, we’re going to use a recursive template definition! The tuple class will instantiate itself recursively, storing the arguments as it goes. Basically the tuple class will become a compile-time linked list of sorts. This means we’ll also have to use more metaprogramming to get access to the elements, but for now let’s take a look at the tuple class definition itself:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

template <typename... Args> struct tuple;
 
// recursion termination condition
template <> struct tuple<> {};
 
// recursive tuple template
template <typename Head, typename... TailArgs>
struct tuple<Head, TailArgs...> : private tuple<TailArgs...> {
        /**
         * \brief  Recursive constructor
        **/
        tuple(const Head& head, const TailArgs&... tailArgs) :
                tuple<TailArgs...>(tailArgs...),
                mHead(head)
        {
        }
 
private:
        Head mHead; ///< Tuple head
};

Although this might look a bit weird at first, there’s not actually that much that’s new here. We’re still just recursively defining a template class, and unrolling a parameter pack as we do it, but we’re doing it via inheritance this time. The tuple class’ base class is actually itself. We use the same trick we’ve used before to extract the first parameter from the parameter pack, but now we pass the template’s TailArgs to the base class so that the base class’ Head becomes the first element in its super class’ TailArgs.

When the instantiation of this class finishes it will be a tuple class that has an mHead member variable set to the value of the first argument in the parameter pack given to the tuple’s constructor as the head variable (which you can see being set at line 14). That tuple class will have a base class that itself has an mHead member variable which is set to be the second parameter in the initial instantiation’s parameter pack (or the first parameter given to the base class instantiation). This process repeats until all the parameters in the initial pack are exhausted and the termination condition is hit.

Okay, so that’s pretty cool, but now what? Basically we’ve got this weirdly constructed tuple class with a strange inheritance hierarchy that stores all the elements given to the constructor — we still need some method for extracting them! This is where all our previous metaprogramming fun comes into play. We’ve already built much of the framework we need to make this happen:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

template <typename Head, typename... TailArgs>
struct tuple<Head, TailArgs...> : private tuple<TailArgs...> {
        /**
         * \brief  Recursive constructor
        **/
        tuple(const Head& head, const TailArgs&... tailArgs) :
                tuple<TailArgs...>(tailArgs...),
                mHead(head)
        {
        }
 
        /**
         * \brief  Get a tuple argument
        **/
        template <int N>
        typename elementType<N, Head, TailArgs...>::type
        get() const {
                return getValueFromTuple<N>::template getValue<typename elementType<N, Head, TailArgs...>::type, Head, TailArgs...>(*this);
        }
 
        /**
         * \brief  Get the head value
        **/
        Head                            head() const { return mHead; }
 
        /**
         * \brief  Get the next argument in the chain
        **/
        const tuple<TailArgs...>&       next() const { return *this; }
 
private:
        Head                            mHead;                                  ///< Tuple head
};
 
// let's use our new get function!
tuple<int, char, bool, double> t1(1, '2', true, 3.14159);
char c = t1.get<1>(); // template get function where the index specified returns that parameter of the tuple class

The first function we’ve added is the tuple’s get function, which can be seen being used at line 37. So how the %@*! does that work anyway?

Our new class member template function works pretty much the same as Boost’s tuple library (although much simpler). All we’ve done is defined a template function that takes an integer as a template parameter, and we’ve used our elementType struct, which we’ve already seen from earlier in the article, to set the return type of the function to whatever type is at that index in the tuple’s parameter pack!

We are missing some pretty important code though. The get function then calls the getValue function, which we’ve yet to define:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

template <int N>
struct getValueFromTuple {
        template <typename ReturnType, typename Head, typename... TailArgs>
        static ReturnType getValue(tuple<Head, TailArgs...>& t) {                                                                          
                return getValueFromTuple<N - 1>::template getValue<ReturnType, TailArgs...>(t.next());
        }                                                                          
 
        template <typename ReturnType, typename Head, typename... TailArgs>
        static const ReturnType getValue(const tuple<Head, TailArgs...>& t) {                                                                          
                return getValueFromTuple<N - 1>::template getValue<ReturnType, TailArgs...>(t.next());
        }                                                                          
};                                                                           
 
// termination condition for when getValueFromTuple is instantiated with zero
template <>
struct getValueFromTuple<0> {
        template <typename ReturnType, typename... Args>
        static ReturnType getValue(tuple<Args...>& t) {                                                                         
                return t.head();
        }                                                                         
 
        template <typename ReturnType, typename... Args>
        static const ReturnType getValue(const tuple<Args...>& t) {                                                                         
                return t.head();
        }                                                                         
};

This might look a bit complicated, but there’s absolutely nothing here we haven’t seen before. We define a template struct called getValueFromTuple that takes an integer as a template parameter. This is done so that we can partially specialize the struct so we have a termination condition for the function we actually need (getValue). C++ function templates are a bit more limited than class templates in how we can partially specialize them — specific specializations by constants are not allowed, so we use this struct wrapping trick to give us what we need.

The getValue function is also nothing new. It does the exact same recursive unrolling that the elementType struct uses to extract a specific parameter by index. The function recursively instantiates itself until N reaches zero and the termination condition is called. The trick here is that getValue calls the tuple class’ next() function during each instantiation, which passes the tuple’s base class to the next instantiation of the function. All we’re doing here is walking the “list” of parameters passed to the tuple’s constructor, we just happen to be doing it by walking the class hierarchy.

Let’s take a quick look at the tuple’s next() function to see how this works:

const tuple<TailArgs...>&       next() const { return *this; }

This is how we trick the compiler into returning its base class — which has an mHead that contains the next parameter in the pack. The return type is the same type that the base class has, so when we return ourselves through the this pointer the compiler does the right thing and casts down to the base class.

So there we go: a fully functioning variadic tuple class that stores its constructor parameters and gives us a way to get at them by their index! We can now do this:

tuple<int, char, string, double> t1(1, '2', "variadics rule", 3.14159);
 
int    var0 = t1.get<0>(); // var0 is now 1
char   var1 = t1.get<1>(); // var1 is '2'
string var2 = t1.get<2>(); // var2 is now "variadics rule"
double var3 = t1.get<3>(); // var3 is now 3.14159

Sweet! Of course there are many improvements that could be made such as allowing access to elements by reference (for efficiently storing and retrieving large structures), adding iterators, copy constructors, operator overrides, etc, but I’ll leave that stuff as reader exercises .

For the next installment the plan is to go even deeper down the the metaprogramming hole, and see what kind of compile-time fun we can have. Template metaprogramming is a really useful skill to have; the more you can do at compile time, the fewer instructions need to get executed at runtime, plus, as we’ve just seen in this article, metaprogramming trickery can make other code more readable and, in turn, easier to maintain. Not only that but playing around with template metaprogramming is a great way to get to know your compiler, and the C++ language!

The same goes for C++’s templates. Have you ever wondered how Boost’s Tuple library works? Well, Boost does a lot of cool complex things, like defining variadic templates using preprocessor and template metaprogramming, but in the end it’s actually pretty simple:

struct NullArg {};
template<typename Type1 = NullArg, typename Type2 = NullArg, typename Type3 = NullArg> 
class tuple {
        tuple(T0 t0) {}
        tuple(T0 t0, T1 t1) {}
        tuple(T0 t0, T1 t1, T2 t2) {}
};
 
tuple<int, char> t(1, '2');

Of course this tuple class doesn’t store or let you retrieve arguments, only allows for up to three parameters (adding more arguments, while monotonous without some preprocessor fun, is easy), and there’s some definite metaprogramming magics that go into tuple’s get<X>() template function, but it’s all still possible.

Getting back to the point: alright so we can, in effect, do variadic templates already, even without fancy C++0x. So why are they good? Well, all we’re really able to do is emulate variadic templates. The compiler always sees those “variadic” structs as having the maximum number of parameters, which means extremely long mangled names (increased binary size), slower compilation time, stupidly complex compiler diagnostic output, and did I mention slower compilation time? Not to mention that it’s an ugly and hacky way to achieve something that could be much more elegant. It’s fine for libraries to define handy structs with variadic emulation, but do you really want your production code nastied up like that? Likely not.

So, in come’s C++0x’s variadic templates. They’re easy, fun, and the whole family can use them! The syntax looks like this:

template<typename... Args> 
class tuple {};
 
tuple<char, int, bool> t;

Not so useful yet, but as you can see the ellipsis operator is given new purpose, and in this case it’s meant to denote a template parameter pack. It can also be used to denote a parameter pack expansion, like this:

template<typename... Args> 
class tuple {
        tuple(Args...) {};
};
 
tuple<char, int, bool> t('1', 2, false);

Here we’ve created a variadic tuple template and expanded the Args parameter pack in the constructor, thus letting us pass a variable number of arguments to the tuple’s constructor.

You can use the pack expansion pretty much anywhere it makes sense: base classes, constructor initializers, you name it, but in my opinion the real power of variadics comes once you start using them in conjunction with template functions and we begin to utilize C++’s powers of argument deduction. Take a look at this:

template<typename... Args>
struct tuple {
        tuple(Args...) {};
};
 
template<typename... Args>
tuple<Args...> make_tuple(Args... args) {
        return tuple<Args...>(args...);
}
 
int main() {
        auto t1 = make_tuple(1, 2, 3, 4, 5);
        return 0;
}

Nice and clean! There’s no need to specify all the template arguments to make_tuple because the compiler deduces them from the function arguments! Also note the use of the C++0x auto keyword there. Auto variables are still statically typed, they just get their type set to whatever initializes them.

So there’s the ultra-basics. Up next I’m going to start exploring what you can really do with variadic templates — like how you can extract an arbitrary element from a pack, pack searching, and other fun compile time trickery. You’d think Bjarne might have wanted to allow random access to variadic parameter lists, but that just wouldn’t be any fun. Why do things with random access when you can do it all with recursion!

Ready for more? Head to Part II!

Prior to a couple weeks ago I had never played with Qt, and I actually had sort of a negative opinion of it. I had known that it required this extra compilation step where it preprocessed C++ source files for some reason… but I’ll back get to that soon — for now I’ll just say I thought it was silly because I didn’t understand it. The other thing I thought was that Qt was just a widget toolkit for building GUI applications.

I was wrong. On both accounts. First of all Qt is so much more than just a GUI API. It’s an amazingly well designed collection of classes for building robust cross-platform apps. Example, the QString class is so much more useful than std::string. It has built in support for all types of unicode strings, and has a ton of handy functions and operators available. Memory management is also really well done in Qt — you can often “new and forget”. It has a high performance computing API that includes out of the box support for things like Map-Reduce, job scheduling, futures, etc. I could go on and on, but suffice to say, Qt is another tool I’m really glad to put into my toolbox right next to Boost.

As a quick aside, before getting to my actual point, I’d like to mention Nokia’s recent acquisition of Trolltech (the company responsible for Qt), and their decision to LGPL the library. I think that this is probably the greatest thing that has happened to Linux, since the Linux kernel itself, and perhaps the GNU tool collection. I honestly can’t see why any developer would continue to support and use Gnome now that Qt is LGPL’ed, and you can use it to write cross-platform apps that use native widgets on Mac, Windows, and Linux. Competition is usually good, but not when it comes to competing desktop widget sets.

Alright, my actual point, and the thing that inspired me to post about Qt in the first place: the MOC. Qt’s Meta Object compiler. What it does is preprocess a C++ header and build another header out of it that you then include into your source file along side the original header. This preprocessing step gleans all of the information from your class(es) and puts it into a format digestible at runtime. Qt has kindly wrapped this up into a nice meta object system that stems from the QMetaObject.

Essentially what they’ve done is given reflection to C++. If you’ve ever programmed in languages like Python, Ruby, Objective-C, C#, or Java, and friends, you know how awesome and useful reflection is. With Qt’s MOC system you can do many of the neat things in C++ that you can in dynamically bound languages. You can enumerate all the methods of a class, and then invoke them via a string representation of their function name. You can dynamically cast to and from super-classes (again, via a string), or iterate over an object’s class hierarchy — all at runtime! It’s awesome. Really awesome.

But, does this extra MOC step require some insane build system? Well, fortunately no! Thanks to the KDE project having adopted CMake as their build system they’ve created all the necessary build scripts for using Qt in your project. You just do your build setup like you normally would with CMake, include the Qt CMake module, which automatically detects the Qt libraries, and sets up support for automoc’ing your source files.

And while I’m on the topic I might as well say a few words about Qt as a GUI library. I’ve use quite a few GUI APIs in my day: everything from raw WIN32, to MFC, and .net’s WPF (Windows Forms), Cocoa and Carbon, WxWidgets, GTK, TK, AWT, Swing, and there’s probably a few others in there that I’ve [likely thankfully] forgotten [or repressed].

In the past, I’ve never really been impressed by a GUI API before. Cocoa is the one I’ve used the least, but I was definitely impressed by it — that being said it’s sort of wrapped up in the Objective-C way of doing things, which isn’t necessarily bad, but it’s a bit more difficult to evaluate on its own because of that fact. Windows Forms is also relatively well done, but nothing to write home about. And the rest are just plain awful. With the shining star being Qt.

When I created my first Qt GUI (through the help of Qt’s excellent GUI designer) I was extremely happy. I’ll be the first to admit that I hate GUI programming, but let’s face it, every once in a while you do need an app with a GUI, and Qt takes all the crap work out of GUI programming. There’s almost never a need to write code that resizes widgets when you resize your application’s window (they have a well thought out Layout system), and the whole process of designing a GUI and including it in your app is very well done.

The GUI designer creates a nice human readable XML file that you can either have your CMake build system auto-process for you and produce C++ code that you merely include into your project, or you can use the Qt library to dynamically load and process the UI XML files at runtime. Who doesn’t want their users to be able to change your GUI around in whatever twisted ways they want? Well okay, probably a lot of people, but at least the option is there!

All in all I give Qt a resounding thumbs up. I will probably use it in every spare-time project I do from now on, even non-graphical ones, just because I think the API is that good — Qt really has given C++ a gift with their meta-object system. And now with Nokia’s support and LGPL goodness, there’s finally no reason not to use Qt.

I, personally, am in the camp that loves design. I have seen some beautifully elegant code written by crazy geniuses <cough>John Carmack</cough> that was… less than well designed, but most of the programs I admire (of which I also have access to the source code) generally have equally admirable design philosophies.

But, this isn’t a perfect world. It’s a bit sad that we still haven’t figured out a way to hook computers directly to our brains, so we’re left with using terribly inefficient and somewhat cumbersome programming languages to wrangle these computation machines into doing something close to what we wish them to. Of course they have the irritating habit of doing exactly what we tell them, rather than what we mean.

I bring this point up because although programmer skill is definitely one issue, languages themselves suffer from design issues, and I’d say a good majority of the time that’s spent struggling with design is actually spent struggling with a language’s design. It’s the same battle artists have fought for years: I’ve got a beautiful image in my head, now how do I get this paint brush to paint that image on this canvas?

For this series of of articles I’m going to pick on C++. Love it or hate it’s so prevalent that even if you’re one who spends most of your time with more modern languages like Ruby, Python, and Haskell, you’re still likely forced to bust out some C++ now and again. And for those of us who spend 40 hours a week (or more) with C++ and love it, I’m sure we’ve all got a fairly lengthy laundry list of problems that don’t necessary come out as clean as we’d like when we put them into the crazy kitchen sink that is C++. So basically, I’m going to pick on C++ because it’s easy — that and the fact that I’m a C++ compiler developer by day, so I may have a somewhat unique perspective on the language.

For this first Effective C++ installment I’m going to start off slow. I’ve got plans to dive into some more advanced things like template metaprogramming and some of the upcoming C++0x features down the road, but I thought first I’d tackle a problem that’s plagued me more than once, and in more languages than just C++ over the years. It’s also an issue that I’ve seen some fairly… wacky (I’m being nice)… solutions to by students and even in some open source code in the past: it’s the joyous design problem known as the cyclomatic dependency.

But before I begin let me just say: there’s no judging here. Say what you want, but I know as well as the next, given certain circumstances sometimes a goto, or a Duff’s device can actually be an elegant solution, so I’m not going to sit here and say “rethink your design”, I’m going to trust that you have, and move on. Fact is, you’ve run into an issue where you need to have class A depend on class B, and class B on A. Let’s look at an example:

Here is NetworkServer.hpp:

#include "NetworkConnection.hpp"
 
class NetworkServer {
        list<NetworkConnection> mConnections;
};

And here’s NetworkConnection.hpp:

#include "NetworkServer.hpp"
 
class NetworkConnection {
public:
	NetworkConnection(const NetworkServer& server) : mServer(server);
private:
	NetworkServer& mServer;
};

You can immediately spot the issue. Now, of course, this simplistic problem could be reworked in any number of ways to avoid the cyclical dependency, but real world problems are often much more complex and thus harder to refactor, yet often come down to some variation of the above.

Fortunately C++ actually has a fairly decent solution to this problem: the forward declaration. It’s simple, and it works well, but before looking at it, let me say that I’ve seen enough work arounds to know there’s plenty of C++ programmers out there who don’t know what a forward declaration is, or if they do, why it’s useful, or what the C++ compiler actually does with it.

One of the solutions I’ve seen to this is to use generic programming, or in C++ speak: templates. We could define NetworkConnection like so:

template <typename T>
class NetworkConnection {
public:
	NetworkConnection(const T& server) : mServer(server);
private:
	T& mServer;
};

This works, but what if someone instantiates NetworkConnection with a type that doesn’t provide the same methods as NetworkServer — broken design! Not only that but if you’re using NetworkConnection as a base class it might not make sense to have it templated because you might need to cast from it to a super class, and you might not know what it was instantiated with at casting time. This is actually a design flaw with C++, and is being addressed with the upcoming C++0x language feature known as concepts (which I intend to look at in detail in a future installment).

There’s also a non-template way to deal with this issue, however, and it’s something I’ve seen used a lot, especially by students (who are, through no fault of their own, ingrained in the java-single-inheritance-heavy-on-the-interface style of design). You simply make class A inherit from an abstract base class that provides pure virtual function interfaces to the methods needed by the other class caught in the cyclical loop. Then pass the abstract class around rather than its implementation and voila, dependency avoided:

NetworkServerInterface.hpp:

struct NetworkServerInterface {
	virtual void someFuncA() = 0;
	virtual bool someFuncB() = 0;
	...
};

NetworkServer.hpp:

#include "NetworkServerInterface.hpp"
#include "NetworkConnection.hpp"
 
class NetworkServer : public NetworkServerInterface {
public:
	void someFuncA();
	bool SomeFuncB();
private:
	list<NetworkConnection> mConnections;
};

NetworkConnection.hpp:

#include "NetworkServerInterface.hpp"
 
class NetworkConnection {
public:
	NetworkConnection(const NetworkServerInterface& server) : mServer(server) { mServer.someFuncA(); }
private:
	NetworkServerInterface& mServer;
};

This works, and can be a good technique to enforece certain design principles (it’s easy to only allow specific objects to use the NetworkServer in exactly the way you want by only providing a subset of functions in the abstract class definition), but it’s not without its shortcomings. Firstly, significant development effort is wasted by requiring the interface class and main class be kept in sync. Then there’s the overhead of virtual function calls that should be considered. For a time critical section of code this scenario should be avoided like the plague.

Another possible solution is the dreaded void *. It’s a holdover from the days of C programming, and much like the goto, is generally frowned upon in modern OO design. The problem with using a void pointer is if you need to use the member variable as its actual type then you have to cast it back and you’re left with the same cyclical dependency because then you need to include the casted type’s header file again.

Fortunately with C++, since the headers no longer include one another, and the translation unit is often broken into separate header and source files, you’re free to include any headers you wish in the parts of the translation unit that contain the implementation definitions (the cpp files). So, you could store the variable as a void pointer and cast from it whenever you need to access its actual type’s methods, ie:

NetworkConnection.hpp

class NetworkConnection {
public:
	NetworkConnection(void* server);
private:
	void* mpServer;
};

NetworkConnection.cpp:

#include "NetworkConnection.hpp"
#include "NetworkServer.hpp"
 
NetworkConnection::NetworkConnection(void* server) {
	mpServer = server;
	static_cast<NetworkServer*>(server)->someFuncA();
}

Yep, this works, but it’s nasty ugly, and kills proper design because you can’t really return the void pointer from public facing functions in any safe way. Normally I wouldn’t even bring this up, but it is an option, and it’s also a precursor to the forward declaration.

Essentially with forward declarations you’re telling the compiler that you want to use a class without knowing anything about it. In compiler terminology basically you’re doing namelookup, and that’s about it. You can’t use any objects of that type until you give the compiler the details it needs. This is fine though, because you can make sure to only use the variable after its class definition is available (ie in out of line member functions, or in the implementation part of the translation unit). Here’s an example:

NetworkServer.hpp:

#include "NetworkConnection.hpp"
 
class NetworkServer {
public:
	int someFuncA() { return 0; }
private:
	list<NetworkConnection> mConnections;
};

NetworkConnection.hpp:

class NetworkServer; // Foward declaration
 
class NetworkConnection {
public:
	NetworkConnection(const NetworkServer& server);
private:
	NetworkServer& mServer;
};

NetworkConnection.cpp:

#include "NetworkConnection.hpp"
#include "NetworkServer.hpp"
 
NetworkConnection::NetworkConnection(const NetworkServer& server) : mServer(server) {
	mServer.someFuncA();
}

Not only does this solve the cyclical dependency issue, but it speeds up compilation times as well. I’d even go as far as to say it’s good practice to use forward declarations wherever possible, which is in most header files if you’re writing a traditional application or a library that’s not meant to be completely header based.

So there you have it, a look at how to deal with cyclical dependencies in C++.

I’m thinking that the next topic will be a multi-part series on the upcoming variadic templates feature of C++0x since you can already play with them via gcc >= 4.3. Plus my goal is to eventually focus on more metaprogramming type stuff, and variadics are going to make that a lot more fun.

First of all the good: the bottom line is, D is a great little language, and I say little because in many ways it’s a lot simpler, and less complicated than any of the other system level object-oriented languages that it competes with (C++). This doesn’t mean that D is any less valuable though — in fact much the opposite. D’s inherent simplicity is by design. It’s pretty clear by looking at the language spec that the idea was to take the best elements of some languages, throw out the rest, and add some of the missing features that we all wish we had (built in strings, optional garbage collection, modern language features, etc).

And by modern language features, this is the kind of stuff I’m talking about: mixins, lambdas, variadic templates, lazy evaluation, delegates, nested functions, anonymous unions, array slicing, contract programming, handy statements. Basically, much of C++0x’s upcoming feature set, and more.

So does D deliver? The answer yes, a resounding yes in fact. In terms of core language design and features, D provides exactly what many of us have been yearning for for years — a system level language that is on par with (or better than) modern scripting languages (such as Python, and Ruby) in terms of development speed, provides the language features we all should have by now, yet compiles to native code that can be as efficient as C++.

The panacea of programming languages? Well… no, not quite. Unfortunately the story doesn’t end there.

The first problem is a major one, and I don’t want to downplay it because as it stands it’s enough of an issue that no development house could even consider D as a prospective language, but fortunately it’s something that should get better over time. The issue: compiler bugs, and plenty of them. Some of them are small, some of them are huge, but there’s enough that they add up to a serious problem.

The bugs I ran into ranged from little things like floating point printing errors, all the way to compiler segfaults. Being that D is so young and the compilers aren’t close to maturity yet, any project complex enough is basically guaranteed to run into problems with either of the existing compilers, and my thesis project was no exception. Fortunately there are two major compilers that use separate optimization paths (a frequent culprit in the official digitalmars compiler) so one at least has a contingency option, but unfortunately, they share the same front-end, so they tend to exhibit some of the same bugs as well.

There’s also some serious performance issues with both compilers. For example, if you look at the assembler output of either of the compilers you can see them doing the same wasteful, redundant memory copies when doing something simple like returning a struct from a function. The only work around to this compiler bug is to do ugly things like return values from functions using by-reference function parameters.

Take a moment to think about the case where you can’t return values from functions — design goes out the window, along with readability and maintainability, not to mention ease of porting to other languages (should the desire arise). It extends further than that though — it means, for example, using custom foreach iterators to do a simple iteration over a collection of stack allocated structs becomes so inefficient that you end up having to put regular for loops in places where they really shouldn’t be.

Basically these “little” compiler issues make it difficult to consistently employ OO design principles, and use all of the nice modern language features that made the language so attractive in the first place. And after discovering these issues, along with their associated workarounds (wasting time discovering that your program’s poor performance is due to the compiler and not your code), you’re basically left with either a slow, inefficient executable, or ugly, unmaintainable code written in a language that was specifically designed to produce clear, concise, elegant code.

It’s a terrible shame to have a few minor problems result in such broad consequences, but these issues are all things that can be fixed, and they’re not nearly as prevalent as this post makes them seem. In fairness, given how new D is, the overall stability and quality of the available compilers is actually quite impressive. The language itself shouldn’t be judged on these compiler issues, but it’s important that people who may be considering adopting D be aware that there are problems with the compilers before forging ahead.

The next big issue isn’t technically related to the language itself, but is still a problem because it’s one of the root causes of much of what’s stopping D from living up to its potential. The heart of the matter is that D is currently going through some growing pains, and project leadership isn’t keeping up with the community.

Right now the D community is split between two incompatible standard libraries (Tango, and Phobos) that offer up different design philosophies, and although the majority of the community uses one library (Tango), it’s not the official standard library. This tends to create confusion for new users, and forces people to adopt one library or another, or library designers to write and maintain two sets of sources — neither being optimal solutions.

To come back to the issue of the compiler bugs — in other projects such a situation might not be as much of an issue, but many of the compiler bugs have been sitting idle in the D bug tracker for many months, and many releases. It appears as though the project is transitioning between a small niche language, to something that might even be getting close to being considered mainstream (D is currently 12^th on the programming language popularity list, and still moving up), so it’s understandable that some project leadership issues might crop up along the way.

Regardless of how understandable these growing pains may be, it’s still something that the project maintainers, and the D user community, are going to have to deal with before the language can really take the next step in terms of public acceptance.

There’s also one final problem with the language, or rather, the upcoming D2 language spec, and as big as all the other issues are this is actually the biggest thing that’s keeping me from continuing to use D on future projects. We all know that multi-core systems are the defacto standard these days, and although we’re starting to get a handle on developing tools to help programmers deal with the almost always non-trivial task of writing parallel code, we’ve barely even scratched the surface of what compilers and runtimes can do to help regular joe-six-pack-programmer effectively, and easily, design algorithms that take advantage of today’s processors.

Given the current state of affairs in terms of SMP and distributed development, and that there’s a new revision of the D language in the works, D has a great opportunity to assert its intent to be a player in the upcoming multiprogramming language war. But nothing is in the works. The D team isn’t actively working on providing any inclusion of a modern parallel programming paradigm for D2, which means it will likely be pushed back to some future release of D. This is unfortunate, because I know a great many developers are waiting for tools to help them deal with parallel programming (even if they don’t know they are).

Aside from Erlang and other more obscure functional languages, right now C++ is still one of the best choices around for parallel development. With the resounding success OpenMP has seen in the last few years, and with C++0x on the horizon and the new OpenMP 3 being recently released it looks like C++ is going to continue to be the language of choice for most system level and computationally expensive tasks.

Neither D, nor the upcoming D2 spec, give developers anything beyond raw threading (plus some critical section synchronization and resource locking type mechanisms, along with a controversial const system), but definitely nothing in the way of a modern parallel programming methodology. It’s another unfortunate oversight that takes a language that should be an ultramodern successor to nearly all current languages in its class and puts it years behind C++, and even Fortran (both of which have OpenMP at their disposal), and let’s not even try to make a comparison to other languages like Erlang, JoCaml, Parallel Haskell, and friends.

I really wish I could recommend D as a natural successor to C++, but given all the issues it’s currently facing, I just can’t. It’s simply not there yet, and I fear, given it’s current direction, that it never will be. I still think it’s a great project, and I hope that it works out because I’d like very much to see it succeed, because I think a strong competitor to the multi-paradigm system-level language would be a great boon to the software development and computer science communities.

I’ll definitely keep watching D to see what happens in the future, but I likely won’t be using it for any more projects until they can iron out the compiler stability and efficiency issues, and implement some reasonable multiprogramming method.

The thing is, garbage collection isn’t a trivial problem, and it’s certainly not gotten to the point where it’s perfect. Cycles aren’t really an issue for modern conservative style GC’s, not like they are with reference counting schemes, but that’s not to say they don’t still cause a problem. Discovering cycles isn’t free, and generally in a complicated memory setup, even with a good fully modern GC you still end up with weak references / pointers like you would using simple reference counting.

I’m writing this because I’ve recently fallen onto the GC bandwagon as I’ve been using D quite a bit lately. Prior to D my only experience with GC was some time spent with Objective-C, and 4 years of undergrad cursing using Java — but that doesn’t really count , and of course a myriad of interpreted languages like Python, PHP, lua, Ruby, etc. I have used the Hans Boehm GC with C and C++ a few times as well, but that was for select cases where the scenarios were a good match for a garbage collected system.

After spending some time really working with a GC language (under situations that are less than an ideal use for a GC system — ie a real-time interactive simulation), I’ve come up with a more rounded opinion of garbage collection, and I thought I ought to share some of the things I’ve discovered that the GC pundits don’t seem to want you to know:

• Managing memory using GC, in complex data structure situations, where weak ref’s and cyclical dependencies are required (pretty much every largish program every written), can be more complicated and time consuming than manual memory management. If someone tells you GC is all about “new and forget”, while it is true much of the time, you’re still okay to give them a good kidney punch because they’ve never really used GC.
• Memory leaks are not impossible with GC. It doesn’t take much to fool a GC system into believing memory should be retained when it shouldn’t. This is a continuation of the above point, as it generally only happens in complicated situations, but nevertheless GC is not the panacea-all-hail-the-mother-of-memory-leak-free-programming that the pundits claim it to be. If programmers can’t be trusted to manage their own memory, and GC systems are easy to fool, what makes us think that programmers should be trusted to manage GC memory either?
• Using GC is slow. “What?! Didn’t you say using GC is more efficient and leads to higher memory throughput?” I did, but I was baiting a bit, because that’s what the propaganda says. Yes using a good, modern, GC leads to higher throughput of memory allocs and frees, this is true. What the GC evangelists don’t remind you of is that most [well designed] programs allocate a bunch of memory, use the same memory over and over again, and then release the memory, so most of a program’s time [usually] isn’t spent allocating and deleting. If your GC app needs to run for a lengthy period of time, chances are the GC’s sweep phase is going to use up more processing power than would have been “wasted” on manually allocating your own memory. Which leads me to the next point.
• GC makes it easy to fall into bad programming habits. This is a bit subjective, and obviously not true in all cases. But in using a GC language I’ve noticed a tendency in myself to new things that shouldn’t necessarily be newed. Since I don’t have to worry about memory management the odd new here or there doesn’t really matter, right? In a non-GC language I would certainly have created a temporary on the stack rather than allocate memory on the heap (which as we all know is orders of magnitude slower than a stack alloc — and it’s even worse in a GC scenario). Basically think of any Java code you’ve ever seen… how many times have you seen something like: GridCell.setPosition(new Position(x, y)) inside a loop? Ack! Granted this is sort of unavoidable in Java, but especially for people who are still learning, this is a terrible habit to pick up.

So given all of this, it sounds like I’m saying GC is complicated, slow, and makes you dumb. But I’m not. I still hold true to the notion that memory management is something a programmer shouldn’t often have to explicitly deal with. GC is a good thing!

But getting the whole story with garbage collection is important. Like any tool, it’s one of those things that has good use cases and bad, and knowing where and when to use it is key. Fortunately, in D’s case, it’s ridiculously easy to mix manual and GC memory management, so for me, it’s not that much of an issue. I just get sick and tired of hearing the all-pro GC arguments, because in reality GC makes life for a programmer a little bit tougher. And by tougher I mean that more knowledge is required to complete a task.

In the end though it’s definitely worth it. GC might not get rid of memory leaks, and it’s probably going to make your program slightly less efficient, and it might even introduce a few “bad” habits, but in some cases these are reasonable trade offs. If you know a temporary heap allocation is going to get collected right away, and you’re not in a tight loop or something, then why not let the GC do some work for you! If using GC ends up leading to some complicated weak reference scheme, do it manually!

Basically it comes down to the age old adage of: know your tools. Garbage collection can save valuable development time by freeing a programmer from having to deal with the bulk of an application’s memory management, thus leaving more time to focus on those situations where you do need to get dirty with some mallocs and frees (or hopefully on things more interesting than memory management). But, if used improperly, or in the wrong situation, it’s pretty easy for GC to end up causing more problems than it solves.