Posted tagged ‘development’

Learning notes from a C++ n00b – Part 1

August 25, 2010

I am primarily a Java developer. My college decided in my sophomore  year to switch from C/C++ to Java for most of the classes. While it did give me a very good foundation in Java development, it created a bit of a gap in my education. While I’ve done work in C in the past, I haven’t had the opportunity to work on a serious project in C or C++. Since my goal is always to learn a language a year, starting now C++ will be that language. It should be exciting.

Get excited by vintage letterpress

I'm excited. Are you excited? You're probably not. You're probably thinking "Why don't you already know C++? What sort of developer are you?" Well to find out, you should keep reading!

This post is the beginning of a series describing my process of exploration and discovery into the wonderful world of C++ from the perspective of a Java programmer. I imagine that some people who know C++ and Java are getting out popcorn waiting for the inevitable meltdown with dealing with things like memory leaks, makefiles, platform differences, and probably a bunch of other common issues I know nothing about. While that may eventually be the case, I’m not there yet.

Immediately, one of the biggest departures between Java development and C++ development is the change in IDE. For Java development I’m a big fan of IntelliJ IDEA. Java has a lot of import statements that involve very long package names and an IDE like IDEA tends to take care of that for you. Refactoring, like changing method or class names, is easy since I don’t need to manually find all usages and change them. Autocomplete and click jumping to method or class definitions is also very handy. The integrated debugger is useful when needed, and I like that it can easily import maven pom files as projects. While I’m definitely a proponent of being able to program in your language without an IDE (you shouldn’t let your IDE program for you), it still can be a useful tool at times.

My new IDE for C++ is vim. Ok, I don’t know if I’d really call it an ‘IDE’ exactly, but it’s my editor. I like vim overall, and I’ve been an avid user of it for about 7 years, so I can maneuver my way around. After reading about ctags, I wrote a script to generate them so that I can use it to jump between functions and methods. I’m pretty sure I’m not using it correctly however, as I have a tags file in each directory and I think the editor doesn’t always pick up all of the tags files I want. I have no idea how to use a debugger, and dealing with import statements is still a pain. Overall though vim is very fast and never hiccups unlike IDEA, so that’s a plus.

Another large departure from Java development is going from maven to makefiles. In Java I can create a maven pom.xml file to list all of my dependencies and plugins. Maven supplies a natural file structure that I can use to arrange my files into packages and place external resources in a place where the classpath will pick it up. There are tools to automatically create stub maven projects that I can just jump into and start coding. I can have any tests in my test directory automatically run just by calling mvn test. I can compile my project using mvn compile, and I can package it all up into a nice jar or war file by using mvn package. These are all default things that come with maven. It takes care of running the javac command to compile my code. It also sets up a testing environment for running tests. A very handy tool.

So makefiles. Makefiles sort of remind me of ant. I never really got into ant because it’s fairly verbose. It requires you to be more explicit in telling the system what to do and how to do it. Really though, makefiles are a bit less useful than ant, as they seem to be more like elaborate shell scripts that use the make utility to run. I still have to put in all of the g++ commands, along with lines of what libraries and other sources to give to the linker. Dependencies have to be accounted for manually. Tasks like clean or test have to be defined along with instructions for what those things actually do. There is no standard setup with clean interfaces for running tests or setting up your file structure. The main positive aspect that makefiles have over maven pom files is that a single makefile can generate several different deliverables (pom files only create a single jar).

I know that there are tools such as autoconf and automake that is supposed to make makefiles more portable to deal with library issues, but they’re not very easy to use. The GNU tool chain has a fairly extensive manual, but it feels like one needs to read the entire thing before even getting started. I haven’t had the time, so the GNU tool chain hasn’t entered my arsenal yet. I’m still not sure how to deal with the library distribution issue, but I’m sure that will come with time.

This is the book I'm currently learning C++ from. It doesn't get into the surrounding tool chain like make or the compiler much, but it's still useful for the language itself.

I’ve only been really looking deeply into C++ for a week or two now, and there are a lot of questions that pop into my mind. I have certain philosophies dealing with programming that I’ve gained over the years. A lot of those practices have been fueled by ideas from the Pragmatic Programmers. Others I have developed on my own. I’m still uncertain as to how to incorporate some of these ideas in my C++ development.

The first and most important of these is unit testing. Java has some very standard unit testing frameworks with JUnit and TestNG. Boost has a library for unit testing C++, and so does Google Test, but I have yet to delve into either of these deeply. Also I’m not sure if there’s a standard way of running tests like you can find with Java. Making sure that each test is in its own section in a makefile doesn’t seem very appealing. Hopefully I’ll find a nice way to easily unit test code (including using mocks), I’m just not there yet.

Also, how do you profile C++ code? I’ve used a tool called yourkit for Java which works very well. I know that there must be profilers for C++, but I don’t know what they are or how to use them. Similarly, debugging is an area that is still very grey. I know that there’s gdb, but I haven’t found a good tutorial on how to use it or how to integrate it into vim or some other editor to set breakpoints and see variable values.

And honestly, while I understand the basic concepts of pointers, I’m still unsure as to when to use them and how to fully utilize their functionality. In Java everything is done by reference (except primitives) and there’s a garbage collector when items go out of scope. When passing any object to a method, it can be modified by that method. I still need to learn a bit more about how scope effects both objects created on the heap and other objects in C++.

Overall there has been a lot of learning in the short time that I’ve been playing around with C++. While I’ve used Objective-C in the past for some iPhone programming, I haven’t really seriously taken on any projects in C++. The learning experience has been great, and quite a bit of fun. While most people would probably argue that makefiles are really annoying to set up, I’ve been enjoying the learning process of figuring out how to use them and creating working builds. My tests have all been small at this point, but they’re sure to grow in size. That said, I know I still have a lot to learn. I shall be delving deeper into the Boost libraries and doing multi-threaded programming with some asynchronous socket connections. I’ll be sure to update again after having gained a bit more experience.


August 16, 2010

Back when I was learning programming and software development in college, I began a practice of learning a new programming language every year. I started with Java, then moved to Perl, C#, Visual Basic, Python, Ruby, Groovy, Scala, PHP… etc. I still think it’s a good practice for every developer. It’s even a recommended practice by many other developers on how to stay in your game. You don’t want to be stuck in a situation later on in life where you have to quickly learn a new language (including new concepts) after you’ve been using the same one for decades. The COBOL programmers learned that about a decade or two ago.

So learning a new language is good. It’s not just good to have the ability to use that language in your current project, which you may not be able to do, but also if you switch projects to one that uses that language. However, even beyond direct use of the language is how learning new ideas and ways of doing things will change your code in your language of choice, or at least the language you have no choice in because it’s the one your project uses.

The Matrix

I doubt that The Matrix was programmed in single language. I bet even an AI uses an abstraction to reduce the processing power needed to design something like that.

Of course, this actually brings me to my second topic about projects themselves. The idea that you should learn a new language every year isn’t really new to many people. However, I think that the idea should be extended further to encompass switching projects every few years as well. In a large project this could mean just switching from one portion of the project to another, but with smaller projects it might mean switching completely to a different project (or at least dividing your time between the two).

The reason behind this is actually similar to why you should learn a new language. By being forced into thinking in the new language and solving problems with a different set of tools, it can increase the breadth of your experience and help you perform better on that original project or ones you face in the future. Similarly, being forced to work on a new project will make you think in new ways about new problems that will help you in old projects and even future ones. If you stay with a single project for too long, you’ll star to stagnate and be unable to really explore and incorporate new ideas that your project’s problem domain doesn’t cover. Lack of exposure to new ideas will prohibit growth. Your resumé won’t be improving, and you won’t be growing as a developer.

Ideally, if you switch projects you would want to switch to one in a different language than your last project. This will help cover both aspects of learning a new language and a new project. While you can always learn a new language in your spare time and on personal projects, you will generally learn faster if you’re thrown in the water as it were. Similar to learning a language by living in a foreign country, learning a new programming language is much easier if you’re thrust into the depths of it with no choice to procrastinate. You have to learn it to work on your project, which will help you gain new insight into development in general.

I’ll be starting this exciting adventure myself as I transition from my current project in Java to a new project in C++/Perl. While I’ve been meaning to for a while, I’ve never done any serious work in C/C++. Being forced into it by having to work on a code base that is primarily in C++ will help facilitate that learning process through experience. I’m excited for the opportunity and I feel that everyone should strive to make those sorts of changes in their work from time to time to keep fresh and productive.

An Idea for Tracer Bullets

August 6, 2010

My favorite software development related book, The Pragmatic Programmer, has a section in it where it mentions tracer bullets. The tracer bullet system they describe really reminds me of the scaffolding system that Rails made popular. It’s sort of a loose structure that gives you something to play with but that really needs to be fleshed out for your finished application. The idea behind using a tracer bullet, or scaffolding, is to have that basic structure to build with and help avoid the ‘blank page’ problem (writers and programmers both know that a blank page is the most difficult thing to write from). So scaffolding is useful, and what Dave and Andy remark as tracer bullets I’m going to call scaffolding for this piece. I would like to introduce a different idea for what tracer bullets are, and how they can help you focus on good Test Driven Development (TDD) and battle the problem of forgetfulness due to complexity.

Book cover for The Pragmatic Programmer

This is an amazing book. My top choice for any developer, especially ones just out of college or with limited 'real world' experience. Buy it, it's worth the price.

If you’ve ever worked on a new system or a larger refactoring of an old system, you’ll have noticed that you get into a sort of groove with what you’re doing. You look at the big picture and you start making your modifications. Something comes up though where you change a method signature and you have to adjust all of the places that use it, which maybe makes you rethink some basic designs and causes another smaller refactor or something during your larger one. Eventually when that’s all done you go back to what you were originally doing. You’re testing all the way which is good, but eventually you get to a point where you stop and say “I’m done”. Or at least that’s the idea. Often it comes out  more like “Am I done?” and you wrack your brain trying to think if you remembered to put everything into place or are you leaving out some small bit that’s going to cause a bug later on. If this has never happened to you and you don’t understand what I’m talking about, you may stop reading.

The main problem seems to be that you have this grand idea at the beginning, and you know pretty much all of the parts that you’re going to have to modify to make things work, you just loose track of what you’ve done and what you haven’t done. Now you can make a paper ToDo list at the start and check everything off, and that’s a viable method. However I would submit that you should utilize your best programming tool right from the start to keep track of your progress: your tests.

When you first collect your refactoring (or basic design) idea, you should know many of the places that you’ll need to modify in order for it to function. Create unit tests for each of these major ideas. These tests should fail with an error message relating to what the idea is and a note that it isn’t actually implemented yet. As you start to refactor, replace these tests with the actual tests for that piece of functionality. You won’t be fully removing the tests (just by having those tracer bullet tests to start with means there’s something that needs to change and those changes need to be tested), but you’ll be adding real tests for your changes to them. When all of your tracer tests are replaced, you should know that you’ve covered everything that your design required.

That doesn’t mean additional testing shouldn’t be done, there are often times areas that need to be modified or added that you did not originally envision. However, it should be a good start to tracking down those areas that need work. It should also prevent you from forgetting any of those key areas that you thought about originally but lost track of during the hectic course of development.

Tracer Bullets in this sense are really just empty failed tests that provide guidance as to what you need to work on. They’re not real tests, but they’re placeholders for where tests should be in the future. They act as a guide to help you figure out what you should be working on next. In addition, since you need to modify those failed tests for your build anyway, it should encourage you to work in a more test driven way by writing your tests first. Since you’ll look at your list of failed tests as your “ToDo” list, it will encourage your first action being to change those tests to something useful rather than a paper ToDo list which encourages you to jump into writing the application code before your tests.

Storm Trooper Helmet

As additional encouragement to practice Test Driven Development, here's the coercive face of failure.

Anything that keeps us more organized and removes unnecessary burdens to our mental flows is a helpful device for a programmer. Being able to utilize any technique that will increase your likelihood to properly test your application code is also a great benefit. This is just an idea I’ve been pondering for some weeks and I’m sure it’s not the only one for combining your design and testing in an up front mannor. If anyone else has any suggestions or practices that they use, please feel free to let me and others know what they are.

Discipline in Coding

June 17, 2010

The title is perhaps slightly misleading. I think overall that coding should be fun. Figuring out how to do something and getting it to work is a mental exercise that can be pretty enjoyable, especially when you finally have something finished that you can get other people to play with. However, when launching into learning a new language, platform, framework, or basically anything outside of your regular development comfort zone, you can lose your focus on doing things the right way. This is where some discipline comes in.

Mr. T Pointing at you. Be Afraid.

I don't know why, but the first image that came to my mind when I thought 'discipline' was Mr. T. So here he is.

By now, we should all be doing certain things whenever we code. Writing unit tests first is important, as is writing good tests that actually prove the working of things. Using version control is a given; even the stuff you mess around with should use version control since it’s hardly any work at all to add it. Building off of interfaces so that you can easily exchange implementations without messing with the code that uses those interfaces (this also goes well with testing). Decoupling your view from your controller from your business logic. These are all great things, but sometimes in the excitement of learning something new we forget about them.

Recently I began learning about Android programming. I started with their tutorials, and used them and the API reference doc to try to create my own simple program. The idea was to create a simple game, something that shouldn’t take too long but that other people could play and have some form of enjoyment from. I decided to create Zilch (also known as Farkle). Since the portion of android that is newest to me is building and manipulating the view, I ended up writing code that was very coupled between the view and the game logic. It reminded me of code I would have written early in my college career. Not my best code.

After the main gameplay was done so that I could see things on the screen and touched them I looked at my code and realized how bad it was. Events tied straight into business logic actions rather than calling a packaged up set of game rules that could be tied to any front end. This means if I wanted to modify my front end to use a different widget for a particular task I would have to change my business logic as well. Not very efficient, and very difficult to work with.

I think this sums it up about nicely. Link goes to the original creator.

It did however help me identify something that makes it difficult to learn a new language or framework: We don’t always port our knowledge across. I’m very able to utilize TDD, separation of concerns, and creating elegant designs that decouple the view from the model and controller, but when jumping into a new language or drastically different platform I tend to devolve to the most simple of programming practices (which aren’t good). Some practices like TDD can, and should, be applied to every project regardless of language. Having come to that realization, I’m going to restart the game that I had been working on by creating a pure backend first in Java and then tying the front end up with the Android specific stuff. I think it will both help in learning android, and in creating a decent implementation of a game.

So when moving to a new territory, try not to forget what you already know and see how it can apply to what you’re doing. There is great carryover between many languages, leverage your strengths and knowledge and strive to write good code, even in your hobby.

20 second builds

June 11, 2010

Back in May of 2007, Linus Torvolds did a talk at Google about git. Specifically he tended to reiterate the point on how your workflow changes dramatically when you can perform a merge or get new code from other people in under a second. Merging multiple branches into your own branch without remembering revision numbers and knowing that your history is in tact, being able to view the entire history of a project without an internet connection, and just being able to work with your version control system in sub-second time can change how you work with your code. You become willing to make branches because it’s so cheap and easy. You can try out new things quickly and have an easy way to toss them if it doesn’t work out, or merge back parts if it does. I think that the philosophy of really fast version control and its effects on your productivity are quite real, and very important. I also think that the same mentality can and should be applied to your build process.

An example of how development might work via a revision control system such as git. I just needed a picture. Image provided by wikimedia commons.

Of course the first thing you might think is “There is no way I’m going to get my build down to under a second.” And for anything non-trivial, you’d be right. However, I think having a goal of a build plus unit tests in under 20 seconds is not an unreasonable demand. Even better if it’s under 10 seconds. But projects can get really large and involve thousands of files and a similar set of thousands of unit tests. How can you build all of that in under 20 seconds? The answer is: you don’t. Do I sound contradictory? I’m not, but you might have to change how you think about your project.

The main idea behind getting a build under 20 seconds is that it will allow you to compile what you’re working on and run all of your unit tests so that you can test your changes and see if anything breaks. If your build takes several minutes you’re not going to want to build and run your unit tests until your done. This makes doing Test Driven Development a lot more difficult because if you don’t run your tests constantly, you’re probably not designing around your tests passing. You might write less tests, have a less elegant design, and ultimately produce poorer quality code. But still, a large project with hundreds of thousands of lines of code isn’t going to even compile, much less run all of the unit tests in under 20 seconds, so what do you do?

As with any development problem, you break it down into a smaller and more manageable chunk. Instead of writing a giant application with thousands of different files and unit tests, you write more small libraries and services that hold maybe a few dozen or hundred files and tests that contain decoupled and independent structures. These smaller libraries and services can have their own separate build process as they are all separate small projects in and of themselves. That way you can write just the unit tests relating to those small projects and cut your build times down to under 20 seconds.

Now granted that’s only the build time for one module, but all of them together can still take several minutes. But that’s ok. Most bugs and development that you’re working on will only touch one or maybe two modules at a time, causing you to run the build process for just those modules to fix the problem or add the feature. There’s no need to compile and run unit tests for a lot of other systems that are completely unrelated to what you’re working on. As for if your change will break things in other packages, that’s what integration tests are for.

Besides, if you separate your code into smaller libraries and services, you’ll be providing a natural way of decoupling systems and making them no longer become dependent on each other. If you write good strong unit tests that mock any external entities that are called or would call your service or library, those tests enforce the contract that your other systems will require. This will ensure that changes you make will not break other systems. If the changes you make do break the systems, you just create or modify your unit tests to fail due to this breakage so that you know why your system should output what it does or call who it does.

By driving down your build times by having sections be in separate projects, you will have faster turnaround and feedback on the work that you are doing right now. You can run a build right after changing a single line of code just to make sure, and 20 seconds isn’t too long of a wait to get a response. It’s almost like a small break, but no quite long enough that you start trolling slashdot and forget you’re building something.

I feel this post needs another picture, so here's my cat Mischief playing with some rope.

From iPhone to Android

June 2, 2010

Starting this friday I’ll be losing my wonderful iPhone and immersing myself into the world of Android with the HTC Evo. The decision has less to do with dissatisfaction with the iPhone or the merits of Android, and more with moving from AT&T to Sprint. My fiancé had been on Verizon and I was on AT&T. Her contract was up and we decided to merge to a family plan of some sort to save money. While originally we were going to move her to AT&T, she didn’t like any of the phones and AT&T was a bit expensive (and a pain to set up). After some research, it became a simple math problem that Sprint won out by saving us $50 a month.

Android Logo

The Android Logo is a little green robot man. Hopefully he will prove to be friendly.

So I’m selling my iPhone (eBay link) and switching to the Evo in order to save some money. However, this does bring up an excellent opportunity as a developer. Mainly in starting to learn about Android development for making mobile applications.

I find the Android development environment quite interesting. I’m not as big of an eclipse fan, preferring IntelliJ IDEA myself, but the Android integration with Eclipse is mighty slick. Having it bring up the simulator was probably as easy as it is for the iPhone. Interesting opportunities arise.

While programming for the iPhone is all done in Objective-C, programs in Android are done in Java or other JVM compatible languages. The ability to use Scala or Groovy for making mobile applications. Granted the integration for Scala and Groovy might not be as good as straight up Java, but it’s still a benefit that it’s available. Also, since I’m a Java developer at my day job, jumping into Android programming should be a lot easier as I will just need to get used to mobile development and not a whole programming language at the same time.

However there are parts of my iPhone I will miss. Since I play Go a lot, I’ve been very involved using a program called Boardz for turn based play. I’m not sure if there’s a good alternative app for the Android (I’m sure I’ll be looking on Friday). However, if there is not it actually provides me with an excellent opportunity. Basically to create my own version for the Android.

You see, I’ve always been interested in doing personal projects, but I’m never able to come up with a good idea. Having an excellently delivered piece of mobile software that will be unavailable on my new platform gives me a perfect opportunity to recreate a similar program for myself and others. I’ve already started a bit by starting a project to convert parts of the Fuego Go library over to Java. Ideally when I have a basic library for dealing with SGF files and game moves along with calculating things like ko and final scores, I’ll be able to plug in a nice Android interface to it and build a server, maybe on App Engine, to handle game events.

If anyone is interested in collaborating on a project like that, drop me a comment. I’ll have some actual code posted to my github project page later today.

Relating Development To Go

May 14, 2010

This is a really niche topic of discussion. Those who have both practiced software development and played/studied Go may appreciate the relation. If you’ve never played go and are not a software developer… well I can’t really help you there. However, today I found the similarities astounding.

There are a lot of terms from Go that are used to describe the game or situations in the game. Things like “lightness” and “flexibility” where your stones are spread out to cover a larger area but with gaps in the middle. The idea being that since they are spread out you’re not too attached to any one stone and can adjust your battle plan by sacrificing a single weak stone rather than being forced to protect a large group of more valuable, and yet still weak, stones. If you cluster your stones together too much, especially at the beginning, you’re said to be over concentrated and are not being very efficient with your stones. Your stones can become “heavy” when they’re too large to lose, but very difficult to keep from being captured. You are forced to make moves that benefit your opponent more than yourself just to make sure you don’t lose those heavy stones.

Japanese style floor goban. Large board tables like this are often made of Kaya, a rare japanese tree known for it's properties as Go board. They are often very expensive.

What does this have to do with software development? Quite a bit actually, especially if you follow the Agile Development Methodology. When creating a software design, you will be more rewarded if you keep your design light and flexible, able to adapt to new change as it happens. The ability to sacrifice (refactor) a weak section will help your greater strategy of winning (producing a complete product). When pieces of your development get heavy, they are difficult to refactor, possibly because they’re not designed to be tested. When modifying that section of your code, it tends to break other sections. Since you built a bad foundation early on and refused to sacrifice it (refactor), you end up being stuck trying to save a dying design and spend a lot of time and effort for little gain.

Once you’ve sacrificed enough bad designs and followed your test driven development to the proper design, your code becomes flexible and more maneuverable. In Go terms, sections of it become “Alive“. Building off of those sections, as long as you keep to your design principles, makes fixing bugs and developing new structures a lot easier.

There are several ways to play go. There is speed Go, where you give yourself only a few seconds to play your piece. There is Go played with a normal time limit, usually an hour or so for each player with some overtime after that. And there’s games which are not timed at all. Speed Go often results in a lot of mistakes, however you can finish a game a lot faster. The result won’t be the best however, and you wouldn’t want to use it to represent your skill in the game. Regular Go is what is good enough for most people. You spend a reasonable amount of time thinking about most moves, but in general you rely more upon any experience you have than reading out the board. In an game that isn’t timed you can really sit and concentrate on your move. You might think about dozens of possibilities of what you could play and the replies your opponent can make and future moves you would make to those replies, etc. It can take a long time, but the resulting moves can be brilliant at times, and often will result in the best game.

I find that too many people play Speed Go in their development efforts. They’re told “Fix this bug” or “Make something that does this” and they open up their IDE or programmer’s editor or echo and cat and put some code in files trying to solve the problem as fast as possible. Problems are dealt with as they come up. There isn’t really a design per say, just how the code all comes together. The problem with this is that the code created by Speed Code is often very problem specific and difficult to fix and modify. There’s no flexibility, and often pieces of the code are heavy.

The last form, coding without a time limit, can yield some surprising results. I’m not saying that you should give everyone unlimited time to code even the most simple of things, nothing would get done. Nor should you try to get the “perfect” design before coding anything, as what you’ll get in the end won’t be worth the time spent. Rather I think that, especially at the start of the project, some actual thought should be put into what you’re coding. This thought shouldn’t be just in your head however, it should be in the form of Tests. If you write up tests that utilize the code you haven’t even written yet, you can get a good design before you code. If it seems hard to test, try something else. You should be writing and throwing away tests as you work on the design of the piece of code you’re tackling. This is equivalent of reading ahead moves in a Go game, seeing where the pieces might lead.

Of course there’s only so far reading ahead will get you. Soon enough you’ll have to implement what you’ve designed and look at the next piece. However, don’t assume just because you have tests and an implementation and that they all work that you can’t change them any more. Everything you write should be flexible to be rewritten or refactored and thrown away or modified to handle new business rules or requirements. Having the tests in place will make modifying the code easier as you’ll see where things will break and can modify them accordingly.

I noticed this correlation in my own code just today. I was finishing up an implementation before moving on to the more difficult part of my project when I realized that from the structure I have built up and the loose framework that I’ve designed, the “difficult” part of the project was able to be solved in the framework that currently existed. The same framework that I’ve built up and torn down and built up several times over the past few months.