We show how the term structure of volatilities for zero-cupon interest rates from the Spanish secondary debt market can be explained by a reduced number of factors. This factor representation can be used to produce time series volatilities across the whole term structure. As an alternative, volatilities can also be derived from a factor model for interest rates themselves. We find evidence contrary to the hypothesis that these two procedures lead to statistically equivalent time series, so that choosing the right model to estimate volatility is far from trivial. The volatility factor model fits univariate EGARCH volatility time series much better than the interest rate factor model does. However, observed differences seem to be of little consequence for VaR estimation on zero coupon bonds.