Ooh. I'm gonna see if I can work this one out. lol.
James having Lily or Ginny's hair would, I'm pretty sure, depend on whether red is more recessive than true black. Bear with me.
Say True black is the dominant: Lily's red hair could be carried by Harry, and then passed to James (and, in fact, Lily II). With red being recessive, none of the weasley's would have any alleles but that, so James would show red hair. If shades of red hair are defined by alleles also, it would simply depend on whether Ginny's hair colour or Lily's is the more dominant. (Biology is very convoluted, though, and I only know a bit, so there are thousands of factors to consider). If shades of hair are not defined by alleles, then this is where my theory runs to an end; I don't know how James II would exhibit one colour over another.
If Red is dominant over true black, however: Lily would have to carry a black allele, because Harry has black hair. He would only have two black alleles. In order for Albus to show black hair (which, canonically, he does) Ginny would have to carry a black allele (I'm inclined to think this theory is the wrong one, because it would mean a recessive black gene being carried through two generations of a completely red-haired family). As long as Ginny passed on her dominant red allele to James II, he would show red hair. In this instance - if alleles determine shades of red - James would not have Lily's hair but Ginny's.
If you want solid answers, searching google should yield quick results. It'll explain the basics of allele pairs, dominant and recessive and which are which when it comes to hair and eye colours. Everything I know is from Equine genetic lesssons, really. I know which eye colours are dominant but I'm not so sure about human hair. just equine coat patterns.
I think that reply was even longer, so again, I'm sorry but I hope it made sense.
If you want to imagine him with dark brown hair (which now you mention, I kind of like) go for it! lol. I personally like(d) to think of both James and Al with black hair, but I realise some people may think it boring.