While at times such notions may not be misguided, this general attitude towards ignoring performance considerations is quite dangerous. Let’s look at a few reasons why performance is still quite relevant, especially early in the development process.

Performance Issues

Some things don’t parallelize

There are a great number of functional behaviours which simply can’t be done in parallel, or rather, gain nothing when done in parallel. At the high-level, most client-server requests tend to have one outermost request which works in serial to assemble the response for the client. Perhaps many of the sub-requests can be done in parallel, but this serial code still has to execute and can often become the bottleneck in the total response time.

At lower levels there are algorithms which can’t be efficiently handled on multiple cores. Often the overhead of splitting up the work is more than the cost of the algorithm itself. Or sometimes, just like the client requests, the algorithm has a serial nature that just can’t be avoided.

Chips aren’t getting any faster

In recent years the speed of individual cores has not really been increasing. While we are certainly not at the theoretical limits yet, the physical obstacles to increasing speed are significant. Essentially we’ve hit a speed limit in the commodity market and chips simply range from 2 to 4 Ghz. Industry has decided providing more cores is better than providing faster cores.

Combine this with the inability to parallelize certain behaviours and you can see a problem.

Can’t fix it later

Designing to be scalable, either to multiple cores, or to multiple computers, is something that has to be planned for fairly early in the process. While you don’t need to scale immediately, you at least need to chose an architecture and algorithms that can be easily adjusted later. Most people, including programmers, tend to think in a serial fashion, thus most code tends not to lend itself to concurrent processing. If you have failed to consider concurrency, and scaling, early in the process, you may find that path fairly difficult.

Even small decisions made early on can lead to significant performance loss when used systemically through the code. Perhaps a band internal flow, or poor use of global memory. Once a bad behaviour is ingrained at all points in the code it becomes extremely time consuming to change. Naturally programmers just follow the existing code; whatever is there at the start will be magnified throughout.

Solutions are non-trivial

This is a good counter to those who blindly argue you should profile your code to see what is more efficient. If any algorithm could be coded in multiple forms within a few hours then perhaps simply trying them out is a good idea. Most algorithms are however part of a larger system and trying to segregate and replace that component can often be difficult. Often the item to be improved can simply not be isolated and is more of systemic feature in the code. Given the amount of time that will be required to make a change, or code the first version, it seems entirely reasonable to try and think about the efficiency ahead of time.

This also entirely neglects that doing proper performance measurements is very hard. But that is a topic all on its own.

Broken theory of scaling

A system which can scale in theory is vastly different than one that can actually scale. A common failure is related to networking. A cluster of computers has to be connected by real switches and wires which have a fixed limit on the traffic they can effectively handle. Beyond that you need to start grouping and segregating. Designs often fail to account for this, requiring direct connectivity between all computers and/or forcing all traffic through a single machine.

Inefficiency costs money

Even if your system can scale simply by adding more computers, this isn’t necessary the best solution. Every additional machine has a real cost associated with it. Beyond an initial purchase and installation cost, there is a continual ongoing cost in maintenance and electricity, as well as the final disposal cost. These expenses are significant, and in a highly competitive service market, reductions of even 5 or 10% can make huge difference in the ability of the company to attract clients.

Conclusion

While spending too much time fine-tuning the details is often fruitless, backing too far away from performance concerns can be downright dangerous. A lot of major performance issues can be handled up-front with just a tiny bit of planning and forethought. Even if your non-functional requirements are light at the start, don’t be surprised by changes in demand, and in particular peak load problems. Never forget that hardware has physical limitations to scaling, and that every additional machine is an added cost.

This is not attempting to be an argument in favour of excessive optimization. It is more of a counter to the alarming trend I see to completely ignore performance issues. Don’t ignore that 5% loss of speed in your module, it’s going to magnify with the 5% loss in all the other code!

Naturally I can’t just condemn such principles without showing why it is a problem. Looking through my own code it is certainly a commonly used keyword. While alternatives do exist, “continue” is an essential aspect of flow control which helps to reduce code and improve clarity. Instead of trying to prove its worth, I’ll instead go the other direction and show the consequences of not using “continue”.

Reductio ad Falsam

Avoiding the minor arguments, let’s jump right to a logical hypothesis: forbidding “continue” also requires forbidding any non-tail “return” statement. That is, if you believe using “continue” is wrong, logically our argument must also forbid the use of using a “return” statement anywhere other than as the last statement of a function. They both prematurely interrupt the flow of execution for a particular block of code.

We can show this using some simple code examples. First let’s start with a very typical loop using the the “continue” keyword.

for( int i : range )
{
if( some_cond( i ) )
continue;

...
}

Now let’s use a function instead of the inline loop body. I have chosen this approach since as it is a common recommendation.

void process( int i, ... )
{
if( some_cond( i ) )
return;

...
}

for( int i : range )
process( i, ... );

By doing this you have not modified the meaning of the code at all. An optimizing compiler could very well compile both code bits exactly the same. You have however increased the size of the code, and moved the loop body logic away from the loop — some people may find this lowers readability. Take this a step further in a language that supports closures (I’ll attempt to use a C++11 lambda):

Ignoring some syntactic bloat, this looks like the original code sample, albeit without a “continue” statement. By using a closure we can meet the requirement yet still produce the same code.

for( int i : range )
[](int i) {
if( some_cond( i ) )
return;

...
} (i);

These code samples are to illustrate that using a non-tail “return” can be identical in meaning to using a “continue” statement. This would imply that whatever reason one would have to forbid the use of “continue” would also apply to this use of the “return” keyword. My intuition is that if I took this line of reasoning far enough the omission of “continue” might actually require a pure functional programming approach. Simply forbidding the use of a non-tail “return” is likely enough to show the absurdity of the original requirement.

Practical Example

By way of example let me show how not allowing “continue” will produce what I consider to be bad code. Looking through my recent project I found a sample that looks somewhat like the below (it scans a list of logical objects looking for those which can be processed).

for( object_id_t id : proc_list )
{
object * obj = find_object( id );
if( !obj )
continue;

if( obj->is_active() )
continue;

time_t elapsed = now() - obj->begin;
if( elapsed < timeout )
continue;

...
}

If we forbid “continue” (and non-tail “return”) this code would have to be convereted using embedded “if” blocks.

for( object_id_t id : proc_list )
{
object * obj = find_object( id );
if( obj )
{
if( !obj->is_active() )
{
time_t elapsed = now() - obj->begin;
if( elapsed >= timeout )
{
...
}
}
}
}

High levels of block depth are difficult to read. It is difficult to visually identify where the block ends and what the chain of conditions is. Converting this to a series of functions would result in a lot of nearly trivial functions. A large number of trivial forwarding functions are also hard to read as it is difficult to follow the code. I find the first code example, using “continue”, to be a very clear way to write the code.

Conclusion

Often requirements get taken out of context, and this might very well be one of those situations. The StackOverflow answer picked one aspect out of a very large document. Out of context, and without rationale I definitely see the requirement (of not using “continue”) to be more harmful than beneficial. When I looked through the requirements, most of them actually have a “Rationale” entry. This is very good practice; you should always document why you have certain restrictions. Strangely however the requirement on “continue” was lacking such a rationale, so we have no idea why it is not allowed — and in particular why they still allow non-tail “return” statements.

First we’ll cover the basics of object lifetime — you may nonetheless wish to read “What’s an Object” as quick referesher on objects. Then we’ll move on to the basics of refernce counting.

Scope Lifetime

The most common lifetime for an object is scope-based: the object exists for the duration of a function or bracketed block of code. Even when using references the name of the reference itself tends to be a scope-based variable.

	int function( int a )
	{
		int b = a + 1;
		a_struct c;

		if( b > 2 )
		{
			int d = b;
			a_struct e;
		}
	}

The above C example shows us basic scope based lifetime. The variable “b” has function scope. When the function is entered an integer is allocated to be referenced by the name “b”. When the function exits this object will automatically be destroyed. The same happens for “c”, but here more space will be allocated as it is a structure and not a primitive. The “if” statement introduces a new scope, and in C the life of “d” and “e” will only be until the end of that “if” block.

It is important to note that virtually all languages have some kind of variable scoping. Even in languages where memory management is automatic, the reference variables still have a scope of some kind. While you as the programmer may not be aware you are working with references, the language does. The basic scope rules play an important role in memory management of all languages.

Pointer Lifetime

For a pointer it is essential to see the difference between the reference (the pointer variable itself) and the refered to object. The actual pointers follow the same scope lifetime rules, in that the pointer object itself disappears once the scope returns, but the refered to object persists.

	int* function( )
	{
		int * a = new int( 5 );
		return a;
	}

	void caller()
	{
		int * b = function();
		delete b;
	}

In this basic C++ example the function creates a new integer object. Inside the function there is a pointer called “a” which refers to the new object. The return statement of “function” creates a copy of the pointer, which wil be assigned to the “b” pointer in the caller function. “b” in turn uses the pointer to delete the underlying object.

This is completely manual memory management. You are responsible for tracking and deleting the object.

Reference Counting

It is often difficult to manually track the life of an object. This is particular true when the object is used widely through the program. It would be helpful to have a somewhat automatic lifetime management. Once everything is done using the object it should be deleted. Reference counting is one such technique.

This method is simply keeping an extra counter along with each object that is created. The counter is the number of references that exist to the object, in the C/C++ case this would how many pointers refer to this object. Anytime a pointer is copied we increment the count, and anytime a pointer goes out of scope, or is reset, we decrement the count. When the count hits zero the object is deleted since nothing more is using it.

Invasive Counting

There are two basic approaches to reference counting, either invasive or non-invasive. Let’s first look at invasive as the examples are slightly clearer. In this approach the objects themselves are aware of the reference counting mechanism. Users of the pointers explicitly increment and decrement the count.

	MyType * a = get_object();
	//do something with a
	a->decrement();

The above is a very typical use of an object with invasive reference counting. You call some function which returns an object. Once you are done with the object you are to release it. Here we call “decrement”; if the count reaches zero the object will be deleted. The called function might look ilke this:

	MyType * current_object;
	
	MyType * get_object()
	{
		current_object->increment();
		return current_object;
	}

The function provides access to some global resource, or more likely a member variable. It calls “increment” prior to return to indicate there is a new reference to the object now. This makes the object safe to use by the caller since the count must be greater than zero after being returned.

To implement this approach all these objects simply need to derive from some common base class. A much simplified form of this class is shown below — many essential details are omitted to demonstrate strictly the core reference counting mechanism.

	class ReferenceCount
	{
		int count;

		ReferenceCount()
		{
			count = 1; //start at 1 as creation implies at least once reference is being made
		}

		void increment()
		{
			count++;
		}

		void decrement()
		{
			count--;
			if( count == 0 )
				delete this;
		}
	};

	//any reference counted object simply derives from the above type
	class MyType : public ReferenceCount { ... }

Microsoft’s COM system uses this approach. The base classes is called “IUnknown” and has the two functions “AddRef” and “Release”.

Non-Invasive Counting

Non-invasive reference counting does not require the objects to derive from any common base type. It is called non-invasive since there is nothing added to the objects for the reference counting, it is handled external to the object itself. This allows reference counting to be added after the fact. The fundamentals remain essentially the same.

	Counter<MyType> current_object;
	
	Counter<MyType> get_object()
	{
		current_object.increment();
		return current_object;
	}

	Counter<MyType> a = get_object();
	//use a.getObject() to obtain the object itself
	doSomething( a.getObject() );
	a.decrement();

This is the same logic as the non-invasive case but simply with a wrapper class. Here MyType isn’t aware that it is being reference counted. One approach is not necessarily better than the other. Each has its own set of advantages and disadvantages.

Still Not Automatic

Neither of the above methods manage to avoid the feeling of being manual. While they certainly help track the use of an object, the programmer is still left calling increment and decrement on their own. How automated this is, or can be, depends highly on the language.

PHP does this completely automatically. All variables are implicitly reference counted. When you assign to a new name in PHP it increments the count, if you call “unset” the count is decremented. The counting is completely hidden from the PHP programmer.

C of course does nothing for you: due to the language itself automatic counting is simply not possible. C++ on the other hand provides a library of smart pointers which can do it automatically.

Example: C++ shared_ptr

C++ offers a shared_ptr wrapper which is a non-invasive reference counter. It simplifies reference counting by doing all the increments and decrements automatically based on assignments and scoping rules.

	shared_ptr<int> function()
	{
		shared_ptr<int> a( new int(5) ); //count is now 1
		return a; //count is now 2, one for "a" and one for return temporary (*)
	} //count drops to 1 as "a" goes out of scope

	void caller()
	{
		shared_ptr<int> b( function() ); //count is now 2, one for tempoary and one for b
		//now count is 1 since the temporary is gone
		shared_ptr<int> c = b; //up to 2 since we have another pointer copy
	} //now count is 0 since "b" and "c" go out of scope, thus the object will be deleted

In the example the current count is shown at each step. The key to note here is that any copy of the “shared_ptr” will increment the count on the object. Once that count hits zero the object will be deleted. In the context of C++ the “shared_ptr” is not a special language feature, but rather part of the standard library. There is nothing magical about it, it is merely a structure similar to below (where T is the object type, usually a template parameter in C++).

	struct shared_ptr
	{
		T * obj;
		int * count;
	};

This reference counted pointer is thus no more than a reference to the underlying object and a reference to the count itself. In addtion it has many operators and functions overloaded to enable the automatic counting mechanics.

(*Some of these steps are merely logical, since an optimizer can actually avoid certain copy operations, like a return statement and assignment from temporary.)

This article assumes you know the basics and tries to expose the lesser known aspects of working with variables. It breaks apart a variable into more specific terms like object, name, and reference. It is by no means a definitive guide, but intends only to expose some underlying concepts often obscured by many languages.

What is an object?

Programming is a way to manipulate data and control hardware. Data tends to be exposed in discrete blocks and hardware via distinct channels. At the most abstract level we can consider both memory blocks and hardware resources under the same umbrella term: object. In an object-oriented language this could be an instance of class, whereas in a lower level language it may be a small structure, or just a single value. For system resources this may be a file, or network stream.

All objects share two things in common: creation and destruction. In order to use an object it has to first be created. This may be a simple declaration, an instantion statement like new, or a system function. Once created the objects are used until they are no longer desired. At this point they are destroyed. Here the variable may smiply go out of scope, the instance explicitly deleted, or a system function called to release the object.

//allocate/free memory in C
char * data = malloc(123);
free(data);

//open/close socket in Flash
Socket sock = new Socket(...);
sock.close();
 

The terms creation and destruction are often used in this context, perhaps coming from the higher level OOP languages. In some cases we actually have more of an acquire/release pattern. The pattern of use is nonetheless identical.

 Names and References

All of those objects we’ve created are not very useful unless we have a way to access them. Every language has a lot of variation in this area, and unfortunately also uses different and overlapping terminology. Syntax aside, there are a few primary ways which we access an object.

By Name

When you declare a variable your compiler creates the backing object for you and lets you access it via the name you’ve given it. This case is so prevalent that often we don’t even consider there to be a distinction; we look at the variable name as though it were the object itself. This name can only ever refer to one object.

//create an integer object, refer to it with the name "a"
int a;
//assign 5 to the integer object referred to be "a"
a = 5;

In the above code you have an integer object created somewhere in memory. In this case you really don’t care where this object is created. You do however need a way to access this data, thus you have also given it the name “a”. It’s tempting to consider “a” to be synonomous with the object itself, but that manner of thinking will cause problems later. You must consider that you have an integer object and the name “a” by which you access it as two distinct entities.

By Reference

The second most common way to access an object is by using some kind of reference, other than a name, to this object. We must be careful in how we use terms since every language does something slightly differently here. It is easiest to start with C since it has a very straight-forward definition of references, which is simply the address of the object in memory.

//C memory references are called pointers
int * b = get_integer();
*b = 5;

The above code seems very similar to our example with “a”, but has some important differences. The type of “b” is an “integer pointer” rather than an integer itself. What we are saying is that “b” is actually the address of some integer. The “get_integer” function gives us this address — we don’t care where it is. The second part is the assignment. Here we are not actually assigning to “b”, but rather to the object pointed to by “b”.

What becomes confusing at this point is that you still have the name “b”, which is a distinct object. “b” is a name for the object which is of type “integer pointer”. You can, like with “a”, assign new values to “b” itself, and thus refer to a new object. “b” has its own memory location. Inside this memory is the address of another memory location.

Examples

Java References

Java references are very similar to C pointers. All class type variables are implicitly references and thus no class instances have any direct names. They are addressed by reference only. For example, here we will compare roughly equivalent code in Java and C++.

//Java
MyClass c = new MyClass();
c.setValue( 123 );</h4>
//C++
MyClass * c = new MyClass();
c->setValue( 123 );

In both classes a new instance of MyClass is being created. “c” is not the object itself, but rather a way to refer to the object. The new instance is actually anonymous, it has no name.

C++ References

On top of C pointers C++ added a reference construct. The standard tries to distinguish a reference from a pointer, but in the end, a reference becomes nothing more than a normal pointer with a slightly different syntax and restricted copy semantics. Here is a common, and misleading example, about what a reference is.

int a; int & b = a;

The classic text at this point is to say that “b” is an alias for “a”. Alias sounds lot like it is referring to the “name” of the variable. This is wrong, C++ references are not established at the name level, but strictly at the reference/memory address level. Consider if our source object is not a named variable.

int * a = get_integer();
int & b = *a;

“b” now refers to the object pointed to by “a”. There is no way at this point at the code to know what “name”, if any , “b” is bound to. It is no longer related to “a” at all; you can freely assign to “a” and what “b” refers to will not change.

File System

Names also exist at the operating system level and though feel slightly different are quite related. For example a file has a particular name on the drive, and this name can be used to locate that file. In this sense we have a named object just as the integer above, though most languages require a special construct to access the file.

First, a basic low-level C example.

//open a file for reading
int out = open( "output.txt", O_WRONLY | O_CREAT );

//write a string to the file
char const * text = "my text";
write( out, text, strlen(text) );

We provide a name to the “open” command and obtain a reference to the resulting file. Note here that the reference is stored in a plain old “int” type. It should be clear that our file is not actually an integer, simply that the file can be referred to via this integer value. That is, the “name” of the file is still “output.txt”, and the name “out” is an integer which stores a reference to the file. Such references are often referred to as “handles”.

Many OS commands can get confusing as to what actually happens. In the “open” example here we’ve actually created two things: the file and the OS file reference. “out” is not a direct reference to the file, but rather a reference to an internal OS structure which in turn knows how to work with the file. So we actually have a reference to a reference. This is clear when you call the “close” command, the file isn’t deleted; just the OS structure is freed, but the file on the disk remains intact.

(This article is part of the series on Defective C++)

A Broken Any Cast

C++ inherited the casting syntax from C. For compatibility reasons, one presumes, the C-style cast is equivalent to either a const_cast, static_cast, reinterpret_cast or combination thereof. Having const_cast and reinterpret_cast in the list make the language dangerous. Like C, the cast notation can convert from any arbitrary pointer type to any other arbitrary pointer type whether they are in any way compatible or not.

This isn’t an argument against the various forms of casting. Each have their own specific purpose and are valuable in their own way. static_cast and dynamic_cast are by far the most commonly used and fulfil most intended meanings of cast. const_cast is frequently used, and though needed, does introduce a potential for invalid code. reinterpret_cast is by far the least used, and the most likely to do the wrong thing. So the failure is that the simplest cast notation can actually invoke the least used and most dangerous casting operation.

Part of the problem is the mixing of casting and conversion. If you wish to convert one type to another type, whether a fundamental or object, you can use a static_cast. This is very bad, since logically a conversion is not the same as a cast: a cast results in a different view on the original object, and a conversion results in a new temporary object. Equating these two meanings just muddles the meaning of both of them. Given that the functional notation also exists for type conversion it is not clear why static_cast needs to do the same thing.

Solution

A saner option would be to do a static_cast or dynamic_cast. Perhaps the sanest option would be to do a dynamic_cast and throw an exception if it could not be converted. Usually when you cast it is because you expect to have the desired type and less frequently because you are checking the type. Thus an exception makes sense here. A distinct operation for type checking is a good option instead of an attempted cast.

For situations where performance is important, static_cast would still be available; dynamic cast is a relatively slow operation. The same rules would apply: you can only static_cast between types part of the same class hierarchy.

Conversion would be done strictly via a conversion syntax and would be unrelated from casting. How conversion is done also has issues, but we can get to those another time.

reinterpret_cast is a difficult beast to approach. It has several very distinct purposes. Ideally a language would have another way to approach each of its uses without having to introduce this very dangerous operator. Exactly how is cause for another article.

Here we present those basic expectations. These rules should be followed by all libraries and deviations properly documented. They represent a sane basis for doing concurrent programming. Any library that does not uphold these requirements, or fails to note deviations, is simply not concurrency friendly or simply not thread-safe.

Data Model

In this discussion we are going to use the term object to refer to any data created by the API. This may actually be a high level language object (a C-style struct, or a Java class) or merely an opaque handle which refers to data. There are other combined representations as well. Regardless of how it is referenced we assume the library exposes discrete data objects.

In this light the library must describe the relationship between its objects. There will be both independent and dependent objects. The requirements listed here focus primarily on dealing with independent objects; logically distinct data structures are the primary focus of concurrency. Note additionally that in some contexts dependents objects may be treated as independent, this facilitates things such as containers and object pools.

Example: A has table is a data object which contains several other data objects. At the level of the hash table itself, such as inserting and removing elements, all objects in the hash table are dependent. However, when working directly on the contained objects, they can be treated as independent of each other.

(These rules apply to both threading and multiple processes equally. They will however be described in terms of threads as that is how most people consider such concurrency.)

Thread Freedom

The library can be used from multiple threads. This is the absolute basic requirement of a threading friendly library. Such a library could yet hardly be called concurrent. The API must allow data operations on objects in different threads at the same time. This is what would be considered the actual minimum to expect from a thread friendly library. It may now be called concurrent as you can actually do data processing in parallel.

An API with only those two requirements would however be troublesome to use. On top of these we need that individual objects are usable from multiple threads. That is, if you create an object in one thread you must be able to use it in another thread as well. Without this ability you, as the user of the library, will be overly stressed trying to use the various objects.

Example: You are using a financial library and wish to do some bond calculations. You create a bond object in thread A and store it in a global table. Later you are in thread B and wish to use that bond object. This should be allowed.

As an aside, certain libraries offer ways to move objects between threads. While this is better than nothing, it is still problematic, especially if you can only push to another thread rather than pulling to your current thread. Some operating systems however have this basic limtiation when it comes to certain resources. Our definition extends to these OS modules as well: they aren’t concurrent, perhaps not even thread friendly.

Dependent Object Read-Write Safety

The standard rules for data access to dependent objects are multiple read safety and exclusive write safety. This equally applies to parallel access to an individaul object (perhaps the most common case of access).

Multiple Read

The functions which do not modify an object (read-only functions) are expected to be completely thread-safe. You can call any number of different read-only functions on dependent objects from any number of threads at the same time.

Example: You have construced a string. Any number of threads should be able to read this string at the same time.

Exclusive Write

Functions which can modify an object require exclusive access to that object’s dependent group. The caller is expected to perform sufficient locking to ensure that none of its other threads are either reading or writing to the same object at the same time.

 Example: An image is actually a collection of related data. You have the bits of the image itself, a color table, author information, print attributes, etc. All this sub-data is child data of the image itself. Therefore it is reasonable that exclusive access (for write) is required on the whole set of data (the image object) when modifying any of this related data.

Independent Object Read-Write Safety

Independent objects must be usable concurrently from multiple threads without restriction or concern for data races. This applies to both objects of distinct types and those of the same type.

Unrelated objects can be modified at the same time. For this reason it is vital the API describe it’s ownership rules and indicate which objects are related to each other. Some APIs may define a parent-child relationship and require that modifications to any children require an exclusive lock on the parent (thereby excluding all reading and writing from the parent and all of its children).

Example: Given a graphics processing library you create image A. The basic requirements state that any thread may read any properties of image A at the same time. If you wish to modify A however you are responsible for providing exclusive access. Now, if you create a second image B, it is assumed independent from image A. In this light you should be able to modify A in one thread and B in another thread at the same time.

Example: Containers usually have a special relationship with their children. The container itself is a single object, thus it requires exclusive access for modifications, including inserting and removing data. However, it is normally expected that so long as the container is not being modified, each member in the container is independent of the others.

Limited Thread Data and Cleanup

Though thread freedom allows objects to be used from multiple threads there is no prohibition on thread local data. Thread local structures can often be used to improve performance by reducing contention on shared data. A library is free to do so, on one condition however: when the thread exits the thread local data must be cleaned up. Simply put, as threads come and go in your program you should expect that the library will not be leaking resources. The space complexity of the library should be related to how many threads are using it now, and how many objects you have.

Related here is how much data, or how many resources, the thread local structures retain. A user of a library should be safe in expecting that only a sensible amount of memory is being used and that no scarce resources are cached per thread. A library which does not uphold this ideal will essentially violate the thread freedom requirement.

Example: You have an audio library for playing and mixing sounds. Some sound cards have on-card memory which can be used for sample playback. This memory is usually quite limited. If each thread in which the libary was used allocated some of this memory for its private use, you’d find it quickly exhausted — and indeed probably sitting idle in threads which aren’t currently using the sound library.

Per-Thread Feedback

Different languages have different ways of reporting error values, in some cases these cannot be returned directly as a result of calling a function. For example, a lot of POSIX functions return a handle-like object, where a 0, or -1, indicates an error has occurred. To actually determine what error has occurred another function, or a global variable (like errno) needs to be consulted.

Such mechanisms must be thread-safe. One thread must be able to call the primary function and then consult the return value without having another thread overwrite that value. In practical terms that means this return state must be thread local.

Example: You call the fopen function to open a file. If file cannot be opened a null value is returned and errno set to indicate what happened. Now, between this call and reading errno another thread also calls a function which sets errno. It is expected that this second call does not disrupt the value of errno in the first thread. That is, each thread has its own unique copy of errno.

External State Defines Requirements

All data objects have an externally visible state tied to some internal state. The user of a library is oblivious to this internal state: they work only with the external view. This gives libraries a lot of implementation flexibility. Since the user does not see this internal model, the concurrency requirements must be described and met according to the externally visible state.

Example: If you have string A and then create a copy of it to produce string B. You should now be able to assume that these are two independent objects with respect to the API. Thus you should be able to modify both objects at the same time from different threads. While this may seem obvious, it has very significant implications for libraries which wish to use copy-on-write semantics.

Visibility

So far we’ve not talked much about low-level data visibility. It is probably too deep for this article, but we must mention it briefly. The standard language rules on visibility apply to all the requirements here. The programmer is expected to create a proper series of happens-before operations and ensure visibility of his objects between threads. This is mentioned in this section since the user will only be ensuring visibility on the external view of the objects. It is vital for the libraries internal state to at least match, or exceed, the user expected visibility.

Low Contention

The rules about thread freedom and data safety are really only useful if the processing can actually be done concurrently. That is, writing to two independent objects can actually make progress on two distinct processors at the same time. It doesn’t help anybody if the first write simply locks the entire library until complete, at which point the second write proceeds.

This is perhaps not a fundamental requirement, but a reasonable expectation of a quality library. The other requirements should not create a lot of resource contention between threads. That is, each call to the library should be able to make progress without waiting on other calls to the library.

Certain APIs will require shared data however, and thus some contention will exist. So what exactly low contention means is a quality issue. On the ideal side we have libraries with zero contention, these tend to be the stateless libraries which actually don’t have shared data. On the really bad side we have libraries which lock on every call and are essentially single-threaded.

Extensions and Exceptions

These requirements are provided as a basic set that a sane library should be providing. If you observe that many libraries don’t meet these requirements you’d be correct. You should also note that many libraries are very difficult to use in a threaded program, some libraries are close to impossible to use correctly, and others end up needing a dedicated and exclusive thread.

A basic set of requirements is still useful when dealing with such libraries. It gives us the basic expectations from which we can note deviations. Rather than just saying the library is not thread-safe we can now indicate specific guarantees which are not met by the library, or which specific limitations are in place.

Other libraries will extend the requirements. The most common extension is creating an active self-locking object. Such objects can be modified by any number of threads at the same time as the library itself will do all the necessary locking. In some contexts this is actually the preferred behaviour, whereas in others it is a significant performance impediment. Again here however, describing a deviation from the expected requirements helps users of the library understand.

(This article is part of the series on Defective C++)

Arrays

The first warning of a memory problem in C++ is the need to match array allocation with special deallocation syntax.

int * a = new int[10];
delete a; //wrong
delete[] a; //right

Any call to `new[]` must be matched with a call to `delete[]`. Since the array in C++ is kind of a dimunitive type, becoming a pointer at any instant, it is not possible for the compiler to enforce this requirement, in which case it’d simply be an inconvenience. Instead you’ll just end up getting undefined run-time behaviour, hopefully your program will crash, but likely you’ll just get unusual memory corruption.

Beyond just a syntax issue, this problem is exemplified by templates: you simply can’t easily write a wrapper that works with both plain pointers and arrays. Look at the shared_ptr wrapper class and you’ll notice it can’t be readily used with an array. This is unfortunate since the pattern of a shared pointer doesn’t change whether the underlying pointer is a single object or an array.

Placement New

An important feature of memory management is the ability to instantiate objects at an already allocated location in memory. This is done with the placement new syntax. Deleting this object however requires an entirely different syntax.

T * a = new (block)T;
a->~T();

Use a custom allocator instead and you’ll also be forced to write  an explicit call to operator delete. While the basic new and delete offer a sane symmetry, the placement syntax is extremely asymmetric and confuses exactly what new and delete are supposed to be doing.

Virtual Destructor

Where the array syntax may be considered annoying, and the placement syntax merely confused, it is hard to deny the ability to delete part of an object is critically flawed. Whenever you delete an object through a pointer to one of its bases, you may either properly delete the whole object, or just delete part of it depending on how it was declared.

struct A { ~A() { } };
struct B : public A { ~B() { } };

B * b = new B;
A * a = b;
delete a; //half-delete

Some might still be shocked to learn the above does not call B::~B but only calls A::~A. Unless the base-class destructor is marked virtual the delete operator will only delete the immediately known type. Here that is an A since it is deleting an A*.

Solutions

The history of these problems stems from C compatibility. In particular, a C-struct uses no more memory than its actual member contents (plus padding for ailgnment). In C, memory allocation and object instantiation are two distinct operations which have to be manually performed. C++  merges these two operations, but does so with some critical flaws..

There is no arguing that at times you’ll need fine control of memory allocation and instantiation. Such options should be provided, but the default should be a sane system where delete just does the right thing. That is, delete A will properly delete an array, single object, or derived object. If the object was allocated via a special allocator it should also properly call the deallocator, or otherwise properly deallocate all via that single delete A syntax.

Doing so may require meta-information to be stored along with any allocated object. Regardless of how an object is created the resulting object must contain enough information to say how it should be deleted. This concept isn’t entirely new, just look at any smart_ptr and you’ll see a deleter function which does exactly this.

Curiously, if you also believe a language should be garbage collected you also implicitly support this feature. The collector will have to know exactly how to destroy any object. Short of a full scanning collector, safe deletion also simplifies the task of any object pool or any variant type for that matter.

(This article is part of the series on Defective C++)

C++ has inherited from C a series of implicit type conversions. On top of this it has added a class-based conversion mechanism. Perhaps this is quite convenient if you don’t care too much about types. Perhaps the concept is even sound, but something is lacking. As implemented they are very counter to the notion of a type-safe language.

For example, integer promotion has the vexing property that adding to short integers results in an normal integer. As does any numeric operation on a char. When trying to work with small values, or use bit-masks, this results in distracting casting to avoid compiler warnings. Warnings which you should not turn off if you care anything at all about static typing.

uint8_t flags = 0;
uint8_t const flag_a = 0x12;

//this is the best syntax:
flags |= flag_a;
//but instead you have to do this to avoid warnings
flags = uint8_t( flags | flag_a );

The craziest conversion involves the bool type. Any pointer implicitly converts to bool, as does a bool convert to any integer type. Explicit conversion to bool inside a conditional is a great convenience, but there is no reason to convert to/from bool by simply calling a function or assigning.

void function( int value, bool flag ) { }

int main()
{
  int a = 123;
  bool flag = true;

  //oops, a common mistake
  function( flag, a );
}

That the above compiles without warning (even with -Wall -Wconversion in gcc) is distressing!

A case of promotion gone wrong appears when you start using the extended integer types. Look at the following code and decide which function is called.

#include <stdint.h>

void func( int64_t a ) { }
void func( double a ) { }

int main()
{
int a = 10;
func( a );
}

Neither are called. This introduces an ambiguity since the int can be promoted to both an int64_t and a double. While promotion to double is a nice convenience, it should be clear that promotion to an integer is to be preferred.

Solutions

Type safety can only really be guaranteed if values cannot be lost by default. Therefore any conversion which truncates a value should never be done implicitly. That is, no casting to shorter or less precise types.

Class constructors should be marked as explicit by default, since the vast majority of single parameter constructors are not intended to be used as converters. I might even be willing to accept an argument that class conversion is never done implicitly and must always be explicitly noted. I might also accept an argument that says no implicit class conversion should be allowed at all.

Bool should be a distinct type, non-integral. In a way it should be treated a condition result, thus allowing all the familiar conversions and avoiding the broken ones.

Explicit casts to whatever you want should of course always be allowed. However it would be good to have clear functions which do this and indicate what will happen. Do you wish to truncate, round, assign to max on overflow, wrap-around, etc?

While all the implicit type conversion may seem to make programming easier, ultimately it just hides defects. These defects are often hard to find as well since they are not of a type that generate warnings or can be identified by run-time analysis tools. The rules for a statically typed language should be strict.

Alternatives

Wherever there is an alternative to goto it should be used. For example, breaking out of multiple levels of nesting can be done with a “continue” keyword which accepts a label for a loop construct. Common exit code from a function can be handled via a “finally” block. Often a goto can be avoided simply by using a closure or local function with a return statement.

Restrictions

Nobody will claim that unlimited goto is ever required. We aren’t trying to recreate setjmp/longjmp, which are truly hard to resolve in a modern language. To preserve stack integrity goto can only ever allow one to go up in the stack. Preserving caller context may also require that goto can only be used within a function.  These limitations are reasonable, and most uses of goto live within these.

Implications

Obviously if the execution path is going to be jumping about very clear rules about variable life-time are necessary. This is the clearest reason why goto can only ever go up the stack. Going down the stack means entering scopes of variables which haven’t yet been initialized. So using a goto will unwind the stack up to the level of the destination. In terms of resources this essentially requires the RIAA pattern for resource management: we can’t allow the goto to leak resources.

In the destination scope it also follows that the construction of any variables is not skipped. (C++ has this rule.) That is, in the scope where the label exists, all variables after that label must have been constructed and initialized prior to the goto statement.

Questions

Jumping outside of the current function probably doesn’t make sense. The most signficant reason is that there is no guarantee that the calling function has the destination label defined. We need to make some allowances however. If the language makes use of anonymous functions, closures and lambdas it may make sense to allow goto in those scopes. If those components are specified elsewhere however, for convenience, or reuse, would goto be allowed?

Examples

Here is a random collection of places where goto may be a suitable option.

Breaking from inner loops

This a common example of where goto is helpful. You have an inner loop and wish to break out of the outer loop. For example, say you wish to find an element in a matrix.

	location_t where;
	for( int i=0; i < num_rows; ++i )
	{
		for( int j=0; j < num_cols; ++j )
		{
			if( matrix(i,j).is_what_we_want() )
			{
				where.set(i,j);
				goto found;
			}
		}
	}
	throw error( "not-found" );

	found:
	//do something with it

There are obviously other ways to doing this to avoid the “goto”, but this code is perfectly clear and easy to follow. It does not make sense to code this another way if the only reason is to avoid using “goto”.

Redoing a block of code

You have a bit of code which you might need to execute multiple times within the same scope. A “goto” can sometimes be the clearest option to express this logic.

	redo:
		...
		if( must_redo_expr )
			goto redo;
		...
		if( another_redo_expr )
			goto redo;

The alternate, and popular approach, is to use a while loop.

	while( true )
	{
		...
		if( must_redo_expr )
			continue;
		...
		if( another_redo_expr )
			continue;

		break;
	};

The loop approach slightly obscures the logic of the code. It also has a severe limitation that if you have an inner loop you simply can’t use “continue” to get to the outer loop. Furthermore, it introduces a new scope which can be a problem for certain variables. Now, you can virtually always use a loop instead, but why would you do that if it requires a lot of code juggling and ends up making the code less readable?

Error handling

Error handling within a function can more complicated than simply returning “false” or throwing a simple exception. Said error handling may depend on numerous local variables.

	int function( int a, int b, int c )
	{
		int d = 0;
		...
		if( detect_problem )
			goto broken;
		...
		if( other_problem )
			goto broken;
		...
		return d;

	broken:
		report( a, d );
		resolve( c );
		return -1;
	}

Repeating the error handling code multiple times would be bad (code duplication is always bad). You might be able to create an external functon, but then you must take care to pass all the variables correctly in each case.

This type of error handling is kind of like a miniature exception handling. You can basically create local “try-catch” conditions without the overhead (both runtime and syntactic) of using exceptions. It keeps your primary logic easier to read by pushing error handling off somewhere else.

State machine

Sometimes the logic of an algorithm is just not amenable to representation as a tree. This quite often appears when doing parsing or encoding.

	void handle_stream()
	{
	idle_state:
		...
		if( detect_high )
			goto high_state;
		if( detect_low )
			goto low_state;
		if( detect_end )
			goto end_state;
		goto idle_state;

	high_state:
		...
		if( continue_high )
			goto high_state;
		goto low_state;

	low_state:
		...
		goto idle_state;

	end_state:
		...
		return;
	}

Though state machines are pretty common we don’t see this type of code very much. Most of such code tends to be created by code generators, such as a parser generator or a protocol generator. In cases where performance is not an issue you’ll instead see a higher level functional, or object based approach — using a “switch” statement or function pointers you can avoid using “goto”, though the logic is the same.