Hi @TSheahan. Thanks so much for joining the forum and for your question.
Quantification is definitely a tricky subject, one that I don't have a perfect answer for. But the answer will depend in part on what you're trying to quantify. If you're looking for a binary answer (express/not expressed) then it's a matter of setting a threshold for your signal on your image, and then counting the positive cells. Now, how to set the threshold? That's partly art. The more stringent you set it, you may have false negatives, but you will gave more true positives. I err on the side of more stringent so I can be sure I'm really seeing true positives. But then of course I'm going to miss real ones too. It's a tradeoff. Since you're dealing with dissociated cells, you should see very little background outside the cells. You can afford to be less stringent perhaps. But you should run the RNAscope negative control probe. This will give you an objective measure of what amount of signal is just nonspecific binding of the non-probe reagents. You could then subtract this level of signal as "background".
RNAscope is quite sensitive, and so you can see a single puncta in a cell or sometimes outside a cell. I noticed with the original multiplex fluorscent kit (v1) there was quite a lot of stray dots outside of cells and it made quantification hard. The new v2 kit, which uses TSA amplification, has very little background and will help things a lot.
Now, if you're looking for quantitative results (not binary, but degrees of expression), then you need to count puncta. For low expressed genes, this can work because you see single or a few copies. But for highly expressed genes (like my example above), there is so much mRNA that you just see clusters. Then quantification is not possible.
For an example of the puncta counting approach in neurons, see this paper from Henry et al: https://elifesciences.org/articles/09800 They also recommend some image-J software.
If you're doing multiple colors, and want to see multiple colocalizations (binary), I don't have a great answer. See this discussion: http://forum.painresearcher.net/t/colocalization-analysis-of-cells-whats-the-best-way-imagej-photoshop/152/3
But @mny3 wrote a sweet MatLab script that kind of does it. he may be willing to share.
I hope that helps. Please share your experiences with us and whatever you learn.