Subjective Tests

• Subjective Video Quality Assessment Methods for Multimedia Applications [1]
• Absolute category rating (ACR)
• Absolute category rating with hidden reference (ACR-HR)
• Degradation category rating (DCR)
• Pair comparison method (PC)

• The method of limits [2]
• In [3], the authors found the JND (Just Noticeable Difference) and JUD (Just Unacceptable Difference) using this method for mixing video tiles with different resolutions.

References

[1] Subjective Video Quality Assessment Methods for Multimedia Applications, ITU-T Recommendation P.910, April. 2008.
[2] George A. Gescheider. 1997. Psychophysics: The Fundamentals. Psychology Press.
[3] Wang, Hui, Mun Choon Chan, and Wei Tsang Ooi. "Wireless multicast for zoomable video streaming." ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 12.1 (2015): 5.

How to determine the reliability of the subjects?

Before analizing the subjective test results, the reliability of the subjects can be determined using the Cronbach's alpha factor [1].


References
[1]  J. L. Cronbach, “Coefficient alpha and the internal structure of tests,” Psychometrika, vol. 16, no. 3, pp. 297–334, Sep. 1951.