This post is a quick tip on how to use the paste( ) function to read and write multiple files. First, let’s create some data.
|
1 |
dataset = data.frame(expand.grid(Trt=c(rep("Low",10),rep("High",10)), Sex=c(rep("Male",10),rep("Female",10))),Response=rnorm(400)) |
The next step is not necessary, but makes the subsequent code more readable.
|
1 2 |
trt = levels(dataset$Trt) sex = levels(dataset$Sex) |
The following example is silly because you would rarely want to split your data as shown in this example, but (hopefully) it clearly illustrates the general idea of using paste( ) to create dynamic file names when writing files.
|
1 2 3 4 5 |
for (i in 1:length(trt)){ for (j in 1:length(sex)){ write.csv(subset(dataset, Trt==trt[i] & Sex==sex[j]), paste(trt[i],sex[j],".csv",sep=""), row.names=F) } } |
The result of this loop is four CSV files: HighFemale.csv, HighMale.csv, LowFemale.csv, and LowMale.csv.
We can use the same basic idea to read those same four files into a single data frame. The key is to initialize an empty data frame and then append, via rbind( ), the data from each of the four files.
|
1 2 3 4 5 6 |
dataset2 = data.frame() for (i in 1:length(trt)){ for (j in 1:length(sex)){ dataset2 = rbind(dataset2,read.csv(paste(trt[i],sex[j],".csv",sep=""))) } } |
I found this approach useful when I used a supercomputer to conduct many, many runs of an agent-based model. My jobs were queued more quickly on the supercomputer if they were small, so I broke my simulation experiments into many small jobs. This produced many files that I needed to combine into one data frame for analysis in R.
Travis Hinkelman
Latest posts by Travis Hinkelman (see all)
- Interactive visualization of survival curves with Shiny - April 4, 2013
- Shiny = Happy People - January 11, 2013
- Good programming practices in R - September 22, 2012
FInally! I’ve asking around for an easy way to accomplish this for a while now in R. I’m glad I stumbled on your post. Now, how to do it in SAS… 😉
Great post! I’ve asking myself how to split my data just like you did in your example! Thank you!
Really appreciate it! Worked perfectly!