A quick break from my usual R-centric posts to toot the horn of one of my other favorite pieces of freeware, NetLogo, which “is a multi-agent programmable modeling environment.” If you are interested in modeling complex systems and spatially-explicit behavior or processes, you should definitely take the time to (at least) download NetLogo and browse a few of the many example models. NetLogo has a relatively shallow learning curve and a supportive community that fields questions from beginners without the snark that you find in many other help forums.
NetLogo suffers from the misconception that it is not a serious research tool and is only useful as a teaching tool or as a platform for quickly building model prototypes. The GUI elements, extensive models library, and emphasis on teaching applications undoubtedly fuel the misconception. However, NetLogo performs well in comparisons with other platforms for building agent-based models. The GUI elements often simplify model development, but you can run also your model headless to improve model performance. I used NetLogo to build a model of animal movement and ran my simulation experiments on a supercomputer, which greatly reduced the computational limitations inherent in running lots of simulations.
You can link NetLogo to Mathematica and R to take advantage of the capabilities of Mathematica and R for analyzing the results of your NetLogo model without “the cumbersome transfer of data via file output and input.” You can also call R functions from within your NetLogo model to, for example, generate random deviates from distributions not included in NetLogo’s primitives. Calling R from within NetLogo carries a speed cost, though. The decision to call R functions from NetLogo therefore involves a trade-off between model execution speed and model development time because of the time to required to write equivalent code in NetLogo.
As you might expect, there are R packages (e.g., simecol) that facilitate building agent-based models in R, but I have not tried them. Because NetLogo is a high-level language specifically designed for building agent-based models, I would expect model development to proceed much more quickly in NetLogo than R (even if you factor in the learning curve for NetLogo). I would love to hear from someone who has developed agent-based models in both NetLogo and R to provide an informed comparison.