The shortIRT R package for developing tests in an item response theory framework

Ottavia M. Epifania\(^{1, 2}\)

\(^1\) Psicostat, Padova \(^2\) University of Trento, Rovereto

2026-09-03

Automatic Test Development

Large validated item banks (\(B\)) and automatic selection of items to obtain \(Q \subseteq B\)

IRT models for the win

Being focused on the item information and on the ability of each item to measure different levels of the latent trait, IRT models provide an ideal framework to find \[Q \subseteq B\]

Automated Test Assembly

Maximin algorithms

Maxmize the minimun measurement precision in specific regions of interest for the assessment provided by test \(Q\)

Minimax algorithms

Minimize the maximum distance from a target function that describes the desired measurement precision of test \(Q\)

Caution

The automatic test development algorithm strongly relies on the assumptions and models used for the item calibration in the item bank!

The shortIRT package

It’s on CRAN!

install.packages("shortIRT")
library(shortIRT)

ITEM BANK \(B\)

\[\downarrow\]

\(Q \subseteq B\)

\[ \swarrow \qquad\qquad\qquad \searrow \]

Specific \(\theta\) levels

Discrete targets

\[\theta_1^*,\theta_2^*,\ldots,\theta_N^*\]

MAXIMIN strategy

\(\theta\)-target procedure

\(|Q_{\text{tt}}| = N\), \(N\) set a priori

Across \(\theta\) range

Target information function

\[\text{TIF}^*(\theta)\]

MINIMAX strategy

Item Selection ALgorithm (ISA)

\(|Q_{\text{isa}}|\) cannot be known in advance

Application examples

Generate \(B\) with dichotmous items

set.seed(1312)
I = 10 
B = data.frame(b = runif(I, -3,3),
                      a = runif(I, .7, 1.5),
                      c = runif(I, 0, .10),
                      e = 1)
theta = seq(-3,3, length.out = 1000)
1
Define the item parameters accprding to the 3-Parameters logistic model for dichotomous items
2
Random values for the latent trait
(a) Item Information Function (IIFs)
(b) Test Information Function (TIF)
Figure 1: Item Bank \(B\)

Set the \(\theta^*\)

define_targets(theta, num_targets = NULL, method = c("equal", "clusters"))
targetsE = define_targets(theta, num_targets = 3, method = "equal")
targetsE
[1] -2  0  2
attr(,"class")
[1] "equal"

theta_target()

theta_target(targets, item_par)
testTT = theta_target(targetsE, B)
summary(testTT)
The item selection is based on the theta-target procedure with equally-spaced targets. 
The procedure selected the following 3 dichotomous items: 
10 2 8 
with parameters: 
            b        a          c e
10  2.2438664 1.390367 0.02130947 1
2  -1.2465959 1.283032 0.02087452 1
8   0.3331977 1.208366 0.09370018 1
These items maximize the information for thetas equal to: 
2 -2 0
plot(testTT)

Set the \(\text{TIF}^*\)

TIFtarget <- tif(iifB, fun = "mean")

isa()

isa(item_pars, tif_target, nmin)
testISA = isa(B, TIFtarget, nmin = 2)
summary(testISA)
The item selection is based on the isa requiring the selection of at least 2 items 
The procedure selected the following 3 dichotomous items: 
5 4 1 
with parameters: 
           b         a          c e
5 -0.1749388 1.0324894 0.04984310 1
4  0.7957328 0.9924159 0.08716286 1
1 -1.0547429 0.9433476 0.01572715 1
plot(testISA)

Additional functions

Function Description
bench() Implement a basic selection procedure
IRT() Compute expected probability for a single item
mpirt() Compute expected probability for multiple items
obsirt() Simulate responses according to IRT probabilities
irt_estimate() Estimate of theta
item_info() Item Information Functions (multiple items, IIFs)
tif() Test Information Function (TIF)

& the methods defined for the S3 classes

Final remarks


In summary

User-friendly tool for developing tests (or short forms of existing tests) in an IRT framework

Works for both dichotomous and polytomous items

Provides different methods for pursuing different aims of test constructions


Drawbacks

Does not provide ALL possible methods for test construction

Bound to the assumptions and models used for the calibration of the item bank

Does not provide any content balancing control