Search the Community

I'm doing a qualitative analysis on survey data, my source file is in excel format. I have coded the whole dataset, and I will have a second coder to code 25% of it so we can have an intercoder reliability measure (the Cohen's Kappa value). We ran a trial in which the other coder coded a few answers and I ran a coding comparison query to see how that goes. Our percentage of agreement was pretty high (99%) but our Kappa value was quite low (0.13) indicating poor agreement. I understand that is "Because the Kappa coefficient calculation takes into account the likelihood of the agreement between users occurring by chance, the value of Kappa can be low even though the percentage agreement is high."(source: https://help-nv11mac.qsrinternational.com/desktop/procedures/run_a_coding_comparison_query.htm#MiniTOCBookMark6) However, I worry that will still be a problem given I coded the whole dataset and a second coder only 25% of it, is there a way I can tell NVivo to use only the 25% of the dataset (only the parts of the data that were coded by both users) to calculate the kappa value? Or is there a way I can "delete" 75% of the data (which was imported to NVivo as a spreadsheet) so that the 25% we want to look at is understood as 100% by the coding comparison query? Also, how can the other coder code the data without seeing my code? I understand they can only omit the coding stripes, but that means that they can't also see what they have coded!

Hello I'm stuck (yet again) on using NV function. I set up an intercoder test and had 2 coders code 48 images(sources) to 27 codes(nodes) and I'm trying to deal with the results of the coding comparison which NV only calculates for every source against every node, resulting in a spreadsheet with 1296 lines of comparison. Exporting the results into excel to try and calculate averages also yields nothing as it continues to return: "ErrorEvaluation of function AVERAGE caused a divide by zero error." I can see formulas [=IF($Q30-$O30=0,1,($P30-$O30)/($Q30-O30))] in the NV example: http://redirect.qsrinternational.com/examples-coding-comparison-nv10-en.htm, however not using Excel very often or being terribly proficient with formulas I'm not sure how to setup and apply a formula to my spreadsheet columns to achieve salient results of Kappa and % agreement? All sources are to be weighted equally. Another issue is when looking at the node reference counts I also can;t see which coder applied which codes to what images. In node view it only shows who coded(modified) what last. I assigned colours to each coder and they also do not show up. Any help would really be appreciated, I'm terribly behind in my work ... Thanks!