More on RELTRAD

Some further thoughts on my post yesterday about Darren Sherkat's critique of the RELTRAD scheme. Before I even begin entertaining work on a statistical defense of the scheme, I think it's worth taking a step back and thinking about basic research design and what we can learn from that about the study of religion. The first step is conceptualization: what concept are we trying to use/understand? One of the advances put forward with the RELTRAD scheme is that it is primarily a measure of belonging, where previous and competing schemes conflate beliefs with belonging.[1] The second step is operationalization: how can we faithfully measure this concept? Ideally, we ask people to which specific congregation they belong. Short of that, we ask people to which denomination they belong. We could not, however, include variables for each denomination in our statistical models, let along variables for each congregation, so we need a way to categorize, to collapse and simplify. To stay true to our conceptualization of religious affiliation, RELTRAD collapses these affiliations into several traditions. These traditions are based on the qualitative work of historians of religion who have identified continuities for formal religious institutions (i.e. churches and denominations) and informal religious movements (i.e. sects and NRM's) as well as on the qualitative work of theologians who have identified several strains of formal scholastic theologies and informal "everyday" theologies. To put it simply, RELTRAD attempts to measure a social reality; there is qualitative evidence that these groups are things and distinct things at that. Previous and competing schemes arguably fabricate the categories that they measure; there is little evidence that there is in lived social reality such thing as a "religious liberal" or a "liberal Protestant" or a "sectarian Protestant." We have conjured these categories for the sake of predictive power in our models completely detached from their own reality. In essence, these schemes come dangerously close to selecting on the dependent variable. And, this is the big deal: we err if we evaluate our operationalization, that is our measures, solely on how well they predict some outcome. The first concern should be whether our measures actually measure something about the social world. We assume sociologically that well-operationalized concepts by definition should be predictive of outcomes so predictiveness does indeed become indirect evidence that we have good measures; predictiveness by itself, however, is not evidence that we have quality measures.[2] Based on qualitative evidence and on the logic of research design, RELTRAD offers the best scheme currently available to social researchers paying attention to religious belonging.

In passing, it's also worth noting that not all Black Protestants are black. In fact, in the cumulative GSS dataset, about 4% of Black Protestants are categorized as non-black.

--
[1] - Yes, RELTRAD does require some imputation using attendance and race measures for datasets that have less-than-perfect denominational or congregational measures (e.g. the GSS). The core of the measures, however, are affiliation.

[2] - Although there is plenty of evidence for the continued social relevance of religious belonging, one could imagine a time when the RELTRAD measures would stop being significant predictors. This would not necessarily mean that the RELTRAD measure were poor or deficient but, instead, that religious affiliation itself has become less socially salient, which would be a major sociologically finding.