Aligning Your Research

Published manuscripts separate portions of research into distinct subsections: introduction, method, results, discussion. This separation -- and the order of these subsections -- implies that each truly is distinct. In other words, first you develop theory, then determine the best method, then figure out how to analyze your data, and finally, draw your conclusions. While it is commonly accepted that research should often be theory-driven (or at least replicated, if empirically-driven!), the best research is developed by integrating theory, methods, and analysis, rather than considering each separately.

Two common issues arise when determining theory, methods, and analysis separately. First, researchers sometimes collect data that doesn't truly test their hypotheses of interest. Second, and more commonly, the data collected are not adequate for a desired analysis. For example, a researcher might collect data from individuals in 5 locations for a multilevel model, then learn afterwards that she doesn't have data from enough locations to estimate the effect of location characteristics on individual-level outcomes. Fortunately, this issues are fairly readily avoided if researchers abide by the following:

1. When developing any part of your project, keep the others in mind. While developing theory, think about how you might collect data and test it. While developing your method, think about your theory - to ensure you are collecting data that allows you to test your theory - and your analysis technique. Considering your analysis technique while developing your method will help prevent you from collecting insufficient or inadequate data for rigorous testing. Finally, when determining your analysis technique, make sure it will actually let you test your theory and that it fits the method you are developing. The most important thing to keep in mind throughout all these stages is that methods and analysis are not separate from theory; they are part of your theory.

2. Have a plan - or a co-author! At times, you will want to test theory that might require you to conduct an analysis you've never used before, or implement a method you are unfamiliar with. When this happens, you will want to either learn the analysis/method in advance, or line up a co-author who knows the analysis/method. Ignorance, in this case, will lead to flawed data collection - a waste of your time and the participants'. As long as you or a co-author knows the method/analysis in question, you can design your study more optimally.

3. Keep it simple. It's hard to resist sexy techniques. Many researchers hear of a new technique - usually one they haven't used before (see 2 above!) - and decide they need to implement it in their research. Sometimes, these 'hot' techniques are exactly what you need to test your theory. Often, however, a simple t-test, ANOVA, or regression will suffice. Ultimately, the best technique is the simplest possible technique that lets you test your theory rigorously.

Are there other techniques you recommend to ensure that theory, methods, and analysis are aligned?