Well, to understand why the covariance matrix of a population is always positive semi-definite, notice that:
\[\sum_{i, j=1}^n y_i \cdot y_j \cdot \operatorname{Cov}\left(X_i, X_j\right)=\operatorname{Var}\left(\sum_{i=1}^n y_i X_i\right) \geq 0
\]
where \(y_i\) are some real numbers, and \(X_i\) are some real valued random variables.
This also explains why in the example given by Glen_b the covariance matrix was not positive definite. We had \(y_1=1, y_2=1, y_3=-1\), and \(X_1=X, X_2=Y, X_3=Z=X+Y\), so \(\sum_{i=1}^3 y_i X_i=0\), and the variance of a random variable which is constant is 0 .
https://stats.stackexchange.com/questions/56832/is-every-covariance-matrix-positive-definite