A new method for analyzing measurement error and reliability for interval-valued survey data under uncertainty

Abstract

To check the accuracy and consistency of the data collected through the survey, two concepts are used: measurement error and reliability ratio. The existing methods, when interval data is recorded under uncertainty, to calculate the measurement error and reliability ratio using classical statistics and the approach based on mid-values of the intervals, cannot be applied appropriately. To overcome these issues in the existing method, this paper presents a novel method to handle interval-valued data while calculating measurement error, descriptive statistics, and the reliability ratio. The proposed method for analyzing interval data considers the degree of uncertainty when evaluating interval-valued responses, which is ignored by the existing methods. A simulated dataset from a questionnaire about the knowledge and use of artificial intelligence tools is used to calculate the measurement error and the reliability ratio. The results show that the outcomes from the proposed method and the existing methods are quite different, and the proposed method is more flexible, informative, and suitable for applying to survey data analysis when recorded under uncertainty.

Keywords

survey imprecise data classical statistics error reliability

1 Introduction

The surveys are conducted to collect information directly from the people living in a certain region. The questionnaires are designed in such a way that the respondents can easily understand the questions and select the suitable response from the given options. Surveys are helpful to get information from the people and to understand their behavior and the real problems they face. During the survey, analyzing the collected data helps identify patterns, trends, or preferences that are shared among participants, as well as between the groups under study. The results of the data collected through the survey help the organizations and government to make policies for the future or to update previous policies. As the sampling method is applied for the collection of the data, therefore, there is no need to study all individuals, and selected people are chosen to collect the data, which makes the survey cost-effective.¹ studied the problems in the estimation and presented the detailed work on the interpretation of the reliability of the data obtained from the survey.² presented a detailed study about the measurement error in the surveys.³ studied the measurement error and sources in household surveys.⁴ studied the methods of estimation of measurement error in surveys.⁵ studied the measurement error in the earnings data.⁶ presented the study using the interval-valued responses.⁷ presented the work on measurement error and reliability in medicine studies.⁸ studied the factors which leads for the measurement error in the surveys.⁹ studied the measurement error using German's longitudinal earnings data.¹⁰ studied the exposure the measurement error in surveys related to modern epidemiology.¹¹ presented the work on the study of public opinion toward artificial intelligence (AI). More details about the knowledge and use of AI can be seen in¹² and.¹³ More information can be seen in^14,15 and.¹⁶

The generalized interval-valued method is used to analyze the imprecise and interval-valued data by considering the degree of uncertainty which is ignored in the existing interval-valued method.¹⁷ The generalized interval-valued methods are an extension of classical statistics and interval-valued methods. The classical statistics cannot be applied when the data is in intervals and the interval-valued methods do not consider the degree of uncertainty even when the data is collected under uncertainty. The use of the existing interval-valued methods by ignoring the degree of uncertainty in the data analysis may mislead the decision-makers.^18,19 worked on the methods to analyze the engineering data using the generalized-interval methods.²⁰ provided several examples to show that the generalized interval-valued method is an extension of the interval-valued method.²¹ applied the generalized interval-valued method in the social sciences.

From the review of the literature on survey sampling, it can be seen that there is a rich literature on the methods of measurement error and the reliability ratio. By exploring the literature and according to the best of the authors’ knowledge, there is no work on the calculations of the measurement error and reliability ratio using the survey data collected from the surveys under uncertainty. The main objectives of the paper are to introduce the methods to calculate the measurement error and the reliability ratio using the survey data by considering the degree of uncertainty. The design of the questionnaire with interval responses will be given for the use and knowledge of the AI tools. This questionnaire will be used to generate the data by simulation. The measurement error and reliability ratio using the proposed method will be compared with the results obtained using the classical statistics and the methods that used the mid values of the interval. The comparative study will be given to show the efficiency of the proposed method for the analysis of uncertain data by considering the degree of uncertainty. The proposed framework is expected to have potential applications in survey design, data analysis, and decision-making processes where uncertainty and ambiguity in the collected responses.

2 Methodology

Suppose the true value is $Y_{N T} ϵ [Y_{L T}, Y_{U T}]$ and the respondent's answer is $Y_{N T} ϵ [Y_{L T}, Y_{U T}]$ . The three degrees-truth ( $T_{N})$ , falsity $(F_{N})$ , and indeterminacy $(I_{N})$ -follow the constraint $T_{N} + F_{N} + I_{N} = 1$ . The neutrosophic representation of the true value and the respondent's answer are given by: $Y_{N T} = Y_{L T} + Y_{L T} I_{N}; I_{N} ϵ [I_{L}, I_{U}]$ and $Y_{N R} = Y_{L R} + Y_{L R} I_{N}; I_{N} ϵ [I_{L}, I_{U}]$ , where $Y_{L T}$ and $Y_{L R}$ denote the determinate components, while $Y_{L T} I_{N}$ and $Y_{L R} I_{N}$ represent indeterminate components. The degree of truth measures how much the respondent's answer overlaps with the target range and is calculated as follows:

T_{N} = \frac{max (0, min (Y_{U R}, Y_{U T}) - max (Y_{L R}, Y_{L T}))}{(Y_{U T} - Y_{L T})}

(1)

Note that if there is no overlap between the respondent's answer and the target range, then $T_{N} = 0$ . The degree of falsity measures the part of the respondent's answer that falls outside the target range and is calculated as follows:

F_{N} = \frac{(Y_{U R} - Y_{L R}) - o v e r l a p}{(Y_{U R} - Y_{L R})}

(2)

It is important to note that if the respondent's answer lies entirely within the target range, then $F_{N} = 0$ , and if it lies completely outside the target range, then $F_{N} = 1$ . The degree of indeterminacy captures the uncertainty in the response and is calculated as follows:

I_{N} = max (0, 1 - T_{N} - F_{N})

(3)

3 Measurement error in survey sampling under uncertainty

In this section, we present the methodology to calculate the measurement error in the survey sampling methods under uncertainty. The measurement error in the survey sampling occurs due to several factors including, for example, misunderstanding the questions, errors in recording the data, complex questions, sensitive questions and errors in coding and analysis. The measurement error is defined as the difference between the target value and the response value. Classical statistics and interval analysis ignore the degree of uncertainty in data, which can lead to wrong conclusions. Neutrosophic statistics considers truth, falsity, and indeterminacy, giving more accurate results from survey data under uncertainty. Suppose that

Y_{N R} = Y_{N T} + ε_{i N}

(4)

The measurement by considering the degree of uncertainty, is given by

ε_{i N} = [Y_{L T}, Y_{U T}] - [Y_{L R}, Y_{U R}]

The measurement error under the uncertainty can be rewritten as follows

ε_{i N} = (1 + I_{N}) (Y_{L R} - Y_{L T})

(5)

The mean measurement error by considering the degree of uncertainty is defined by

{\bar{ε}}_{i N} = \frac{1}{n} \sum_{i = 1}^{n} ε_{i N}

(6)

The measurement error based on the classical statistics part and the indeterminate part is given by

{\bar{ε}}_{i N} = \frac{1}{n} \sum_{i = 1}^{n} ε_{i L} + I_{N} (\frac{1}{n} \sum_{i = 1}^{n} ε_{i L})

(7)

Note that ${\bar{ε}}_{i N}$ is the extension of the measurement error $ε_{i L}$ under classical statistics.

3.1 Variance in survey sampling under uncertainty

In this section, the variance in the survey sampling under the uncertainty will be presented. By following the properties of the variances described in the last section, suppose that the true variance of $Y_{N T}$ is $σ_{T N}^{2}$ and the error variance is $σ_{ε N}^{2}$ . The variance of the variable $Y_{N R}$ will be derived as follows

V a r (Y_{N R}) = V a r (Y_{N T} + ε_{i N}) = σ_{T N}^{2} + σ_{ε N}^{2}

(8)

The variance of true values based on the classical statistics part and the indeterminate part is given by

V a r (Y_{N R}) = (σ_{T L}^{2} + σ_{ε L}^{2}) + (σ_{T L}^{2} + σ_{ε L}^{2}) I_{N}

(9)

The variance of the measurement error under the proposed method is given by

s_{ε N}^{2} = \frac{1}{n - 1} \sum_{i = 1}^{n} (ε_{i N} - {\bar{ε}}_{i N})^{2}

(10)

The variance of measurement error based on the classical statistics part and the indeterminate part is given by

s_{ε N}^{2} = \frac{1}{n - 1} \sum_{i = 1}^{n} (ε_{i L} - {\bar{ε}}_{i L})^{2} + \frac{1}{n - 1} \sum_{i = 1}^{n} (ε_{i L} - {\bar{ε}}_{i L})^{2} I_{N}

(11)

These variances are the extension of the variances under classical statistics. These variances reduce to the variances under classical statistics if there is no uncertainty in the data or the data has precise information.

3.2 Reliability ratio in survey sampling under uncertainty

The reliability of the ratio in survey sampling under uncertainty will be derived in this section. The reliability ratio is denoted by $ρ$ and is defined as the ratio of the variance of the target true variable to the variance of the measurement error. The reliability ratio plays an important role in survey sampling and in calculating the measurement error. The main objective of the reliability ratio is to quantify the variation in the recorded variable that reflects the target variable. The existing reliability ratio has the deficiency that it does not incorporate the degree of uncertainty when dealing with uncertainty. The uncertain reliability ratio denoted by $ρ_{N} ϵ [ρ_{L}, ρ_{U}]$ is expressed by

ρ_{N} = \frac{σ_{T N}^{2}}{σ_{T N}^{2} + σ_{ε N}^{2}}

(12)

The reliability ratio based on the classical statistics part and the indeterminate part is given by

ρ_{N} = \frac{σ_{T L}^{2}}{σ_{T L}^{2} + σ_{ε L}^{2}} + \frac{σ_{T L}^{2}}{σ_{T L}^{2} + σ_{ε L}^{2}} I_{N}

(13)

The uncertain reliability ratio is interpreted as: when $ρ_{N} = [100, 100]$ reflects the no measurement error, and when $ρ_{N} \leq [100, 100]$ reflects that the survey data has the measurement error.

4 Design of questionnaire under uncertainty

In this section, the design of the questionnaire under uncertainty will be presented. The questionnaire related to the use of artificial intelligence (AI) will be given, where the options in each question will be given in intervals rather than the precise options. In many real-life surveys, especially those related to emerging technologies, respondents often provide incomplete, approximate, or uncertain answers rather than exact or precise information. Therefore, the questionnaire should be designed in such a way that this uncertainty can be captured during the data collection phase. The proposed questionnaire allows responses in interval from while accounting for the degree of uncertainty, which cannot be effectively handled using classical statistical methods. Thus, the questionnaire provides a practical example to illustrate how survey data can be analyzed under uncertainty. The objective of the questionnaire is to evaluate the knowledge and use of the AI tools while considering the uncertainty, ambiguity and partial knowledge while conducting the survey. Another objective is to assess the frequency, the amount of time and the knowledge of the AI tools and applications. The questions related to evaluating the use and knowledge of AI tools and applications are reported in Table 1. The twenty questions have four choices: A, B, C and D, with responses in the interval rather than a single value and therefore the data naturally considers the uncertainty. When the responses are intervals, the use of the existing statistical methods is unable to consider the degree of indeterminacy. In addition, the use of the mid-values of each interval may mislead or give different results for decision-making. In addition, it is important to note that the results from the interval-analysis for such responses do not reduce to the results of classical statistics when there is no uncertainty. Therefore, the proposed generalized interval-analysis by considering the degree of indeterminacy when the responses are inherent have the uncertainty can be applied for the analysis of the data. The use of the proposed generalized interval methods presents a more realistic interpretation of the data in the interval for human judgment about the use and application of AI tools and applications. By collecting the imprecise data from the respondents, the aim of the study is to seek to capture the overall experience of the people related to the use and knowledge of the AI tools and applications.

Table 1.
Questionnaire related the use of AI under uncertainty.

Questions A B C D Target range

Number of AI tools use regularly [0,2] [3,5] [6,8] [9,10] [4,8]

Number of hours use in a week for AI applications [0,2] [3,5] [6,10] [11,15] [5,10]

Knowledge in AI using 10-point scale [1,3] [4,5] [6,8] [9,10] [4,8]

Encountering AI generated materials on the daily basis [0,2] [3,5] [6,10] [11,13] [5,10]

Confidence in classifying AI generated materials [0,2] [3,5] [6,8] [9,10] [4,8]

Confidence in making unbiased decision using AI generated materials [0,2] [3,5] [6,8] [9,10] [4,8]

Confidence in securing data while using AI applications [0,2] [3,5] [6,8] [9,10] [4,8]

Percentage of decision-making using AI applications in percentages [0,25] [26,50] [51,75] [76,100] [30, 70]

Number of visiting privacy guideless of AI tools [0,1] [2,3] [4,6] [7,9] [2, 6]

AI decisions are fair? [0,2] [3,5] [6,8] [9,10] [4, 8]

Number of hours save using AI tools in a week [0,2] [3,5] [6,8] [9,10] [3, 7]

Replacement of jobs (out of 10) using AI tools [0,2] [3,5] [6,8] [9,10] [3,7]

Sureness in creating jobs using AI tools [0,2] [3,5] [6,8] [9,10] [4, 8]

Outputs are improved by AI tools [0,20] [21,40] [41,60] [61,80] [30, 60]

Inequality in getting benefits using AI [0,2] [3,5] [6,8] [9,10] [3, 7]

Relaxation in using AI tools [0,2] [3,5] [6,8] [9,10] [4, 8]

Feel worrying using AI tools [0,2] [3,5] [6,8] [9,10] [2, 6]

Number of hours spent in learning AI in a month [0,2] [3,5] [6,8] [9,10] [3, 7]

Chance of participating AI in future work [0,2] [3,5] [6,8] [9,10] [4, 8]

Do you support AI ethical regulation? [0,2] [3,5] [6,8] [9,10] [4, 8]

Questions	A	B	C	D	Target range
Number of AI tools use regularly	[0,2]	[3,5]	[6,8]	[9,10]	[4,8]
Number of hours use in a week for AI applications	[0,2]	[3,5]	[6,10]	[11,15]	[5,10]
Knowledge in AI using 10-point scale	[1,3]	[4,5]	[6,8]	[9,10]	[4,8]
Encountering AI generated materials on the daily basis	[0,2]	[3,5]	[6,10]	[11,13]	[5,10]
Confidence in classifying AI generated materials	[0,2]	[3,5]	[6,8]	[9,10]	[4,8]
Confidence in making unbiased decision using AI generated materials	[0,2]	[3,5]	[6,8]	[9,10]	[4,8]
Confidence in securing data while using AI applications	[0,2]	[3,5]	[6,8]	[9,10]	[4,8]
Percentage of decision-making using AI applications in percentages	[0,25]	[26,50]	[51,75]	[76,100]	[30, 70]
Number of visiting privacy guideless of AI tools	[0,1]	[2,3]	[4,6]	[7,9]	[2, 6]
AI decisions are fair?	[0,2]	[3,5]	[6,8]	[9,10]	[4, 8]
Number of hours save using AI tools in a week	[0,2]	[3,5]	[6,8]	[9,10]	[3, 7]
Replacement of jobs (out of 10) using AI tools	[0,2]	[3,5]	[6,8]	[9,10]	[3,7]
Sureness in creating jobs using AI tools	[0,2]	[3,5]	[6,8]	[9,10]	[4, 8]
Outputs are improved by AI tools	[0,20]	[21,40]	[41,60]	[61,80]	[30, 60]
Inequality in getting benefits using AI	[0,2]	[3,5]	[6,8]	[9,10]	[3, 7]
Relaxation in using AI tools	[0,2]	[3,5]	[6,8]	[9,10]	[4, 8]
Feel worrying using AI tools	[0,2]	[3,5]	[6,8]	[9,10]	[2, 6]
Number of hours spent in learning AI in a month	[0,2]	[3,5]	[6,8]	[9,10]	[3, 7]
Chance of participating AI in future work	[0,2]	[3,5]	[6,8]	[9,10]	[4, 8]
Do you support AI ethical regulation?	[0,2]	[3,5]	[6,8]	[9,10]	[4, 8]

4.1 Descriptive analysis of neutrosophic components

In this section, we present the descriptive analysis of neutrosophic components, namely the degree of truth, degree of falsity, and degree of indeterminacy. The averages of these components are computed using the questionnaire given in Table 1, where a target range is specified for each question. For illustration, consider the first question where respondents selected option B = [3, 5], while the target range is [4, 8]. The overlap is calculated as $o v e r l a p = min (5, 8) - max (3, 4) = 1$ , which leads to the neutrosophic measures $T_{N} = 1 / 4 = 0.25$ , $F_{N} = (2 - 1) / 2 = 0.5$ and $I_{N} = 0.25$ . By applying this procedure, the averages of these three components are obtained for 200 respondents across 20 questions and reported in Table 2. For example, for the first question, the average values of truth, falsity, and indeterminacy are 0.1963, 0.6075 and 0.1963, respectively, indicating that approximately 19.63% of responses align with the target, 60.75% fall outside the target, and 19.63% reflect uncertainty or ambiguity in the responses.

Table 2.
The average of the three degrees.

Questions $T_{N}$ $F_{N}$ $I_{N}$

1 0.1963 0.6075 0.1963

2 0.168 0.79 0.042

3 0.1863 0.5 0.3138

4 0.184 0.77 0.046

5 0.2038 0.5925 0.2038

6 0.2038 0.5925 0.2038

7 0.1913 0.6175 0.1913

8 0.2493 0.5846 0.1662

9 0.18 0.52 0.3

10 0.1913 0.6175 0.1913

11 0.1738 0.6525 0.1738

12 0.18 0.64 0.18

13 0.1863 0.6275 0.1863

14 0.232 0.6337 0.1343

15 0.1875 0.625 0.1875

16 0.1775 0.645 0.1775

17 0.1375 0.725 0.1375

18 0.1913 0.6175 0.1913

19 0.1613 0.6775 0.1613

20 0.1713 0.6575 0.1713

Questions	$T_{N}$	$F_{N}$	$I_{N}$
1	0.1963	0.6075	0.1963
2	0.168	0.79	0.042
3	0.1863	0.5	0.3138
4	0.184	0.77	0.046
5	0.2038	0.5925	0.2038
6	0.2038	0.5925	0.2038
7	0.1913	0.6175	0.1913
8	0.2493	0.5846	0.1662
9	0.18	0.52	0.3
10	0.1913	0.6175	0.1913
11	0.1738	0.6525	0.1738
12	0.18	0.64	0.18
13	0.1863	0.6275	0.1863
14	0.232	0.6337	0.1343
15	0.1875	0.625	0.1875
16	0.1775	0.645	0.1775
17	0.1375	0.725	0.1375
18	0.1913	0.6175	0.1913
19	0.1613	0.6775	0.1613
20	0.1713	0.6575	0.1713

5 Measurement error and reliability ratio using simulated data

The main objective of the study is to present the methods to calculate the measurement error and reliability ratio using the responses with uncertainty. The simulation procedure is conducted as follows: first, a sample of 200 respondents is generated. For each respondent, a true score representing the target value is assigned. Next, the responses are generated in interval form according to the options given in the questionnaire in Table 1, which reflect in the uncertainty in the respondents’ answers. The respondents are asked to select the suitable choice from the four given options and by fixing the true score for each respondent is fixed for the calculation of the measurement error and the reliability ratio under uncertainty. The use of the simulated data gives somewhat accurate results as they do not consider factors such as the respondent's bias and missing responses. Although the main aim of the study is to present the method using the simulated data in the calculation of the measurement error and reliability ratio, the proposed framework can be used in the real analysis by collecting the data from the underlying population and by following the proposed methods in calculating the measurement error and reliability ratio.

6 Uncertainty-Based data analysis

Using the simulated data, the statistical analysis under uncertainty is performed on the data collected from 200 respondents. The respondent's data is recorded along with the target true values for the calculations of the measurement error and the reliability ratio analysis. The descriptive statistical analysis performed and the results of the average values of the target values $[{\bar{Y}}_{L T}, {\bar{Y}}_{U T}]$ , the averages of measurement error $[{\bar{e}}_{L R}, {\bar{e}}_{U R}]$ , the variance for target values $[σ_{L Y}^{2}, σ_{L Y}^{2}]$ and the variances of measurement error $[σ_{L e}^{2}, σ_{U e}^{2}]$ are also reported in Table 3. From Table 3, it can be noted that the values of the average and variance are reported in intervals rather than as single values. For example, related to the question about the number of AI tools used regularly, it is found that the average of the target values is [5.3315, 7.312]. It means that the average of the average number of AI tools used regularly is from 5.33 to 7.31. The measure error related to this question is [-0.2015,-0.377]. It means that the mean of the measurement error ranges from −0.2015 to −0.3777. These negative values show the respondent underestimates the target value and it is from 20.15% to 37.7%. The variance for the target variable is from 0.099 to 0.094. Similarly, the variance for the measurement error is from 0.099 to 0.094. The values of the reliability ratio for the proposed method are reported in Table 4. From Table 4, for the same question about the use of a number of AI tools on a regular basis, the reliability ratio is [0.0129, 0.0148] and the percentage of this reliability ratio is [1.29, 1.48]. For the question about the use of the number of AI tools regularly, the reliability ratio is lower and it is from 0.0129 to 0.0148, which clearly indicates that lower reliability is related to the use of AI tools regularly.

Table 3.
Neutrosophic error and variance of the artificial data.

Questions $[{\bar{Y}}_{L T}, {\bar{Y}}_{U T}]$ $[{\bar{e}}_{L R}, {\bar{e}}_{U R}]$ $[σ_{L T}^{2}, σ_{L T}^{2}]$ $[σ_{L e}^{2}, σ_{U e}^{2}]$

Number of AI tools use regularly [5.3315,7.312] [-0.2015,-0.377] [0.099,0.094] [7.61,6.29]

Number of hours use in a week for AI applications [6.2925, 8.27] [-0.8725,0.34] [0.1002,0.0986] [10.9528,17.0456]

Knowledge in AI using 10-point scale [6.288,7.2925] [-0.74,-0.2825] [0.0807,0.1002] [5.5102,4.9542]

Encountering AI generated materials on the daily basis [6.2925,8.3065] [-0.8075,-0.1465] [0.1002,0.0988] [10.8163,10.9817]

Confidence in classifying AI generated materials [6.2925,7.315] [-1.3725,-0.58] [0.1002,0.1008] [7.4286,6.1378]

Confidence in making unbiased decision using AI generated materials [5.27,7.315] [-0.11,-0.365] [0.0986,0.1008] [7.6803,6.2214]

Confidence in securing data while using AI applications [5.2925,7.315] [-0.1175,-0.335] [0.1002,0.1008] [7.1231,5.7510]

Percentage of decision-making using AI applications in percentages [40.315,60.306] [2.055,6.194] [0.1008,0.0995] [602.98,591.75]

Number of visiting privacy guideless of AI tools [2.2925,3.3225] [1.3025,1.7325] [0.1002,0.0892] [4.4580,5.1128]

AI decisions are fair? [6.315,7.315] [-1.155,-0.375] [0.3175,0.3175] [2.8621,2.5518]

Number of hours save using AI tools in a week [4.315,6.29] [0.425,0.2975] [0.1008,0.1002] [7.2954,6.0168]

Replacement of jobs (out of 10) using AI tools [4.3375,6.2925] [0.8375,0.6325] [0.1003,0.1002] [8.6436,6.9118]

Sureness in creating jobs using AI tools [5.27,7.2925] [-0.185,-0.3975] [0.0986,0.1002] [7.2058,5.8063]

Outputs are improved by AI tools [40.315,50.29] [-6.115,3.007] [0.1008,0.1002] [329.80,322.56]

Inequality in getting benefits using AI [4.315,6.315] [0.695,0.5] [0.1008,0.1008] [7.5522,5.9774]

Relaxation in using AI tools [6.315,7.2925] [1.26,-0.4325] [0.1008,0.1002] [7.3667,6.0193]

Feel worrying using AI tools [2.325,3.301] [2.76,3.51] [0.0964,0.0897] [7.4488,6.1716]

Number of hours spent in learning AI in a month [4.27,6.29] [0.89,0.6525] [0.0986,0.1002] [2.8199,2.4506]

Chance of participating AI in future work [6.315,7.2925] [-1.065,-0.2625] [0.1008,0.1002] [7.9038,6.0848]

Do you support AI ethical regulation? [6.2925,7.2925] [-1.0425,-0.2675] [0.1002,0.1002] [7.09,5.38]

Questions	$[{\bar{Y}}_{L T}, {\bar{Y}}_{U T}]$	$[{\bar{e}}_{L R}, {\bar{e}}_{U R}]$	$[σ_{L T}^{2}, σ_{L T}^{2}]$	$[σ_{L e}^{2}, σ_{U e}^{2}]$
Number of AI tools use regularly	[5.3315,7.312]	[-0.2015,-0.377]	[0.099,0.094]	[7.61,6.29]
Number of hours use in a week for AI applications	[6.2925, 8.27]	[-0.8725,0.34]	[0.1002,0.0986]	[10.9528,17.0456]
Knowledge in AI using 10-point scale	[6.288,7.2925]	[-0.74,-0.2825]	[0.0807,0.1002]	[5.5102,4.9542]
Encountering AI generated materials on the daily basis	[6.2925,8.3065]	[-0.8075,-0.1465]	[0.1002,0.0988]	[10.8163,10.9817]
Confidence in classifying AI generated materials	[6.2925,7.315]	[-1.3725,-0.58]	[0.1002,0.1008]	[7.4286,6.1378]
Confidence in making unbiased decision using AI generated materials	[5.27,7.315]	[-0.11,-0.365]	[0.0986,0.1008]	[7.6803,6.2214]
Confidence in securing data while using AI applications	[5.2925,7.315]	[-0.1175,-0.335]	[0.1002,0.1008]	[7.1231,5.7510]
Percentage of decision-making using AI applications in percentages	[40.315,60.306]	[2.055,6.194]	[0.1008,0.0995]	[602.98,591.75]
Number of visiting privacy guideless of AI tools	[2.2925,3.3225]	[1.3025,1.7325]	[0.1002,0.0892]	[4.4580,5.1128]
AI decisions are fair?	[6.315,7.315]	[-1.155,-0.375]	[0.3175,0.3175]	[2.8621,2.5518]
Number of hours save using AI tools in a week	[4.315,6.29]	[0.425,0.2975]	[0.1008,0.1002]	[7.2954,6.0168]
Replacement of jobs (out of 10) using AI tools	[4.3375,6.2925]	[0.8375,0.6325]	[0.1003,0.1002]	[8.6436,6.9118]
Sureness in creating jobs using AI tools	[5.27,7.2925]	[-0.185,-0.3975]	[0.0986,0.1002]	[7.2058,5.8063]
Outputs are improved by AI tools	[40.315,50.29]	[-6.115,3.007]	[0.1008,0.1002]	[329.80,322.56]
Inequality in getting benefits using AI	[4.315,6.315]	[0.695,0.5]	[0.1008,0.1008]	[7.5522,5.9774]
Relaxation in using AI tools	[6.315,7.2925]	[1.26,-0.4325]	[0.1008,0.1002]	[7.3667,6.0193]
Feel worrying using AI tools	[2.325,3.301]	[2.76,3.51]	[0.0964,0.0897]	[7.4488,6.1716]
Number of hours spent in learning AI in a month	[4.27,6.29]	[0.89,0.6525]	[0.0986,0.1002]	[2.8199,2.4506]
Chance of participating AI in future work	[6.315,7.2925]	[-1.065,-0.2625]	[0.1008,0.1002]	[7.9038,6.0848]
Do you support AI ethical regulation?	[6.2925,7.2925]	[-1.0425,-0.2675]	[0.1002,0.1002]	[7.09,5.38]

Table 4.

Reliability ratio from the proposed and the existing method for the artificial data.

	The proposed method		Using mid values
Questions	$[ρ_{L}, ρ_{U}]$	$%$ of $[ρ_{L}, ρ_{U}]$	$ρ$	$%$ of $ρ$
Number of AI tools use regularly	[0.0129,0.0148]	[1.29,1.48]	0.0248	2.48
Number of hours use in a week for AI applications	[0.0091,0.0058]	[0.91,0.575]	0.0029	29
Knowledge in AI using 10-point scale	[0.0144,0.0198]	[1.44,1.98]	0.0093	0.93
Encountering AI generated materials on the daily basis	[0.0092,0.0089]	[0.92,0.89]	0.0041	0.41
Confidence in classifying AI generated materials	[0.0133,0.0162]	[1.33,1.62]	0.0106	1.06
Confidence in making unbiased decision using AI generated materials	[0.0127,0.0159]	[1.27,1.59]	0.0097	0.97
Confidence in securing data while using AI applications	[0.0139,0.0172]	[1.39,1.72]	0.0055	0.55
Percentage of decision-making using AI applications in percentages	[0.0002,0.0002]	[0.02,0.02]	0.00001	0.01
Number of visiting privacy guideless of AI tools	[0.0220,0.0171]	[2.20,1.71]	0.0105	1.05
AI decisions are fair?	[0.0122,0.0152]	[1.22,1.52]	0.0084	0.84
Number of hours save using AI tools in a week	[0.0136,0.0164]	[1.36,1.64]	0.0068	0.68
Replacement of jobs (out of 10) using AI tools	[0.0115,0.0143]	[1.15,1.43]	0.0064	0.64
Sureness in creating jobs using AI tools	[0.0135,0.0170]	[1.35,1.70]	0.0067	0.67
Outputs are improved by AI tools	[0.0003,0.0003]	[0.03,0.03]	0.0002	0.02
Inequality in getting benefits using AI	[0.0132,0.0166]	[1.32,1.66]	0.0092	0.92
Relaxation in using AI tools	[0.0135,0.0164]	[1.35,1.64]	0.0080	0.80
Feel worrying using AI tools	[0.0128,0.0143]	[1.28,1.43]	0.0059	0.59
Number of hours spent in learning AI in a month	[0.0122,0.0164]	[1.22,1.64]	0.0060	0.60
Chance of participating AI in future work	[0.0126,0.0162]	[1.26,1.62]	0.0078	0.78
Do you support AI ethical regulation?	[0.0139,0.0183]	[1.39,1.83]	0.0081	0.81

7 Comparative studies

In this section, the comparative studies across the results of the proposed method, classical statistics, and the results using the mid-values of the interval will be presented for each question. First of all, the results of the proposed method will be compared with the results from classical statistics, and then with the results obtained by using the mid-values of each interval.

7.1 The proposed method vs. Classical Statistics

This section presents the comparison of results obtained from the proposed method and classical statistics using the questionnaire is given Table 1. As mentioned earlier, the proposed uncertainty-based method is an extension of classical statistics. The results from the proposed method reduce to the results of classical statistics when there is precise information. The results in Tables 3–4 reduce to the classical statistics results (lower values) when there is no uncertainty. For example, for the question related to the regular use of the AI tools, the uncertainty-based mean of the target, measurement error, variance of the target values, and measurement error are expressed as ${\bar{Y}}_{N T} = 5.3315 + 5.3315 I_{N}; I_{N} ϵ [0, 0.37]$ , ${\bar{e}}_{N R} = - 0.2015 - 0.2015 I_{N}; I_{N} ϵ [0, 0.87]$ , $σ_{N T}^{2} = 0.099 - 0.099 I_{N}; I_{N} ϵ [0, 0.05]$ and $σ_{N ε}^{2} = 7.61 - 7.61 I_{N}; I_{N} ϵ [0, 0.17]$ . From the analysis, it can be seen that the proposed method gives the results in intervals with a degree of uncertainty. The results from the proposed method reduce to the results to classical values $5.3315$ , $\; - 0.2015$ , $\; 0.099$ and $7.61$ when there is no uncertainty. In addition, the proposed method provides the information about the indeterminate part, which is 1.97, 0.01, 0.004 and 1.29, respectively. These indeterminate values determine the upper bounds of the interval, extending the classical values present. Figure 1 shows the plotted target mean of the lower values (classical statistics) and the upper values (indeterminate) of each question in the questionnaire. This figure indicates that the target mean curve from classical statistics is lower than the curve representing the indeterminate values. From this comparative study, it is clear that the proposed method produces interval-based results with the degree of uncertainty and the explicit indeterminate values for the question related to the regular use of AI tools. This analysis clearly shows that the proposed method is quite suitable, flexible, and more informative than classical statistics. Therefore, the use of the proposed method can be preferred for the collection and analysis of interval data obtained under uncertainty in survey studies.

Figure 1.

The target mean for lower and upper recorded values.

7. The proposed method vs. method using Mid-Values

This section presents a comparison of the results obtained from the proposed method and the existing method that uses the mid-values of the intervals. The reliability ratio results from both the proposed method and the mid-value method are reported in Table 4. The results in Table 4 show that the reliability ratio values from the two methods differ substantially for each question. For example, for the question related to the regular use of the AI tools, the uncertainty-based reliability ratio from the proposed method is $ρ_{N} = 0.0129 - 0.0129 I_{N}; I_{N} ϵ [0, 0.33]$ , while the reliability ratio obtained using the mid-values of the intervals is 0.0248. This comparison shows that the reliability ratio computed using the mid-values does not fall within the interval [0.0129, 0.0148], which represents the exact bounds for the interval-analysis under the proposed method. For other questions as well, the reliability ratio obtained from the mid-value method is even lower than the lower bound (i, e., classical statistics results). The results using the mid-values are shown in Table 5. The same trends can be observed in Figure 2. From Figure 2, it is clearly visible that the curve of the reliability ratio based on the proposed method either lies below the lower bound or exceeds the upper bound of the interval data. This analysis indicates that using the mid-values in survey data analysis may mislead decision-makers. The reliability ratio estimates derived from the mid-values frequently fall outside the valid range of results, making it impossible for decision-makers to reach an accurate conclusion if they rely on the existing mid-value-based method. From the comparison study, it is evident that the proposed method for interval data analysis is more appropriate than the method based on the mid-values. The proposed method provides the results within the correct range by considering the degree of indeterminacy something that the existing mid-value methods fail to capture.

Figure 2.

The reliability ratio from the proposed method and using mid-values.

Table 5.

Neutrosophic error and variance of the artificial data using mid values.

Questions	${\bar{Y}}_{T}$	${\bar{Y}}_{R}$	$\bar{e}$	$σ_{T}^{2}$	$σ_{R}^{2}$	$σ_{e}^{2}$
Number of AI tools use regularly	12.64	6.03	−6.61	0.18	6.84	6.97
Number of hours use in a week for AI applications	7.28	7.02	−0.27	0.04	13.79	13.70
Knowledge in AI using 10-point scale	6.79	6.27	−0.51	0.21	2.24	2.26
Encountering AI generated materials on the daily basis	7.29	6.82	−0.47	0.04	10.22	10.48
Confidence in classifying AI generated materials	6.80	5.82	−0.97	0.07	6.70	6.71
Confidence in making unbiased decision using AI generated materials	6.29	6.05	−0.23	0.06	6.84	6.87
Confidence in securing data while using AI applications	6.30	6.07	−0.22	0.03	6.24	6.33
Percentage of decision-making using AI applications in percentages	50.31	54.44	4.12	0.05	598.6	597.2
Number of visiting privacy guideless of AI tools	2.80	4.32	1.51	0.04	4.69	4.68
AI decisions are fair?	6.81	6.05	−0.76	0.06	7.16	7.26
Number of hours save using AI tools in a week	5.30	5.66	0.36	0.04	6.47	6.55
Replacement of jobs (out of 10) using AI tools	5.31	6.05	0.73	0.04	7.49	7.68
Sureness in creating jobs using AI tools	6.28	5.99	−0.29	0.04	6.36	6.41
Outputs are improved by AI tools	45.30	43.75	−1.55	0.05	326.09	326.11
Inequality in getting benefits using AI	5.31	5.91	0.59	0.06	6.68	6.68
Relaxation in using AI tools	6.80	5.95	−0.84	0.05	6.55	6.60
Feel worrying using AI tools	2.81	3.51	0.70	0.03	6.17	6.29
Number of hours spent in learning AI in a month	5.28	6.05	0.77	0.04	6.55	6.86
Chance of participating AI in future work	6.80	6.14	−0.66	0.05	6.69	6.89
Do you support AI ethical regulation?	6.79	6.13	−0.65	0.05	6.13	6.14

8 Concluding remarks

The existing methods to calculate the measurement error and reliability ratio under classical statistics and using the mid-values of the intervals are not appropriate for analyzing data recorded under uncertainty. In this paper, the mathematical framework to calculate the measurement error and the reliability ratio was presented under an uncertain environment. Simulated data were generated to apply the proposed method to calculate the descriptive statistics and reliability ratio. Using the simulated data from the questionnaire, the implementation of the proposed method is given and the results were compared with the existing methods under classical statistics and the method using mid-values of the intervals. The results show that the proposed method is more efficient than the existing methods in terms of flexibility and information retention. The results obtained from the mid-value method are not reliable as they are not within the range of the data. The proposed method cannot be applied when all survey data are exact, because it relies on uncertainty to calculate truth, falsity, and indeterminacy. In addition, the proposed method is illustrated using the simulated data for illustrative purposes; collecting real data through an actual survey may be a fruitful avenue. Based on the results obtained from the study, it can be concluded that the proposed method can be applied for the calculation of the measurement error and reliability ratio while conducting surveys in political science, economics, environment and many other fields where the uncertainty is expected.

Footnotes

Acknowledgements

The author is deeply thankful to the editor and reviewers for their valuable suggestions to improve the quality, presentation and novelty of the paper. The authors acknowledge the use of ChatGPT solely for improving the English language, clarity, and readability of our own writing in the manuscript.

ORCID iD

Muhammad Aslam

Funding

The author received no financial support for the research, authorship, and/or publication of this article.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Alwin

. Problems in the estimation and interpretation of the reliabDity of survey data. Qual Quant 1989; 23: 277–331.

Ziegel

. Measurement errors in surveys. Technometrics 1993; 35: 100.

Kasprzyk

. Measurement Error in Household Surveys: Sources and Measurement. Household Sample Surveys in Developing and Transition Countries 2005.

Zahedian

Saba

. Measurement error estimation methods in survey methodology. Appl Appl Math 2016; 11: 97–114.

Jenkins

Rios-Avil

. Measurement error in earnings data: replication of Meijer, Rohwedder, and Wansbeek's mixture model approach to combining survey and register data. J Appl Econom 2021; 36: 474–483.

Ellerby

Wagner

Broomell

. Capturing richer information: on establishing the validity of an interval-valued survey response mode. Behav Res Methods 2021; 54: 1240–1262.

Mokkink

Eekhout

Boers

, et al. Studies on reliability and measurement error of measurements in medicine – from design to statistics explained for medical researchers. Patient Relat Outcome Meas 2023; 14: 193–212.

Celhay

Meyer

Mittag

. What leads to measurement errors? Evidence from reports of program participation in three surveys. J Econom 2024; 238: 105581.

Schmillen

Umkehrer

Wachter

. Measurement error in longitudinal earnings data: evidence from Germany. J Labour Market Res 2024; 58: 8.

10.

Russell

Hunter

Maldonado

, et al. Survey of practices of handling exposure measurement errors in modern epidemiology: are the best practices in statistics being adopted by epidemiologists. BMC Med Res Methodol 2025; 25: 198.

11.

Novotny

Weber

Kern

, et al. Measuring public opinion towards artificial intelligence: development and validation of a general AI attitude short scale. AI & Soc 2025. https://doi.org/10.1007/s00146-025-02478-5

12.

Al-Rousan

Ayasrah

Yahya

SMS

, et al. Design and psychometric evaluation of the artificial intelligence acceptance and usage in research creativity scale among faculty members: insights from the network analysis perspective. Eur J Educ 2025; 60: e12927.

13.

Timothy

. AI-driven fabrication of healthcare survey data: methods, motivations, and ethical implications. Ethics Behav 2025: 1–22. https://doi.org/10.1080/10508422.2025.2552777

14.

Rocci

Varriale

Luzi

. Total process error: an approach for assessing and monitoring the quality of multisource processes. J Off Stat 2022; 38: 533–556.

15.

Garbarski

Dykema

Yonker

, et al. Improving the measurement of gender in surveys: effects of categorical versus open-ended response formats on measurement and data quality among college students. J Surv Stat Methodol 2025; 13: 18–38.

16.

Murray-Watters

Zins

Sakshaug

, et al. Averaging non-probability online surveys to avoid maximal estimation error. J Off Stat 2025; 41: 700–724.

17.

Smarandache

. Introduction to neutrosophic statistics, sitech and education publisher, Craiova. Columbus, Ohio, USA: Romania-Educational Publisher, 2014, pp.123.

18.

Chen

, et al. Expressions of rock joint roughness coefficient using neutrosophic interval statistical numbers. Symmetry (Basel) 2017; 9: 123.

19.

Jiang

Cui

. Scale effect and anisotropic analysis of rock joint roughness coefficient neutrosophic interval statistical numbers based on neutrosophic statistics: Infinite Study; 2018.

20.

Smarandache

. Neutrosophic Statistics is an extension of Interval Statistics, while Plithogenic Statistics is the most general form of statistics (second version): Infinite Study; 2022.

21.

Martínez

Hidalgo

Matos

, et al. Neutrosophy for survey analysis in social sciences. Neutrosophic Sets Syst 2020; 37: 409–416.