Lab 5: Classification Accuracy Assessment

Goals:

The main objective of this lab is to introduce methods in which the accuracy of a classified image is determined.  Accuracy assessments will be done to classified images. Accuracy assessment is very important when publishing classified data.  If the image has poor accuracy it is not ethical to publish and should be classified in a more through manner.


Methods:
The accuracy assessment will be completed on the unsupervised image that was created in lab 3.  For more information please refer back to lab 3.  The first step to finding the accuracy of the classification is to collect reference samples.  This was done by using the accuracy assessment add random points tool in ERDAS.  The following image shows the settings used for generating random points.

In total 125 points were created and stratified random sample was used.  This means a certain number of points will be included in each of the classification classes.  This minimum was set to 15 points for each class.  The following image shows the random points on the reference image.

The next step was to go through and determine what land cover each point is in on the reference image.  This was done by showing ten points at a time and zooming in on them and filling out what category they are in on the accuracy assessment table.  The following image is of point number 1 and the corresponding table where the actual land cover class was recorded.  Once the classification was recorded, the point turns yellow so the user knows they already did that point.  The process was repeated for all 125 points.

The next step is to produce the accuracy report.  This is done by using the produce accuracy report tool in ERDAS.  This is simply a button in the accuracy assessment table that creates a report.  From this report, an error matrix and accuracy reports are transferred to an excel table to be more visually appealing.  The resulting tables will be displayed in the results section of this blog.

The last step to this lab was going through the same accuracy assessment process but with the supervised classification image created in lab 4.  A comparison of the accuracy's will be showcased in the results section.


Results:

The unsupervised image got a accuracy percentage of 48.80%.  This turns out to be very poor.  This is likely the result of using unsupervised classification with only 5 classes and maybe because this is the first time the user has done this.  There could have been error in both the classification portion of this process but also when conducting the accuracy assessment.  Mostly, the unsupervised classification method does not allow the user to customize their classes so they cannot train the computer on what pixels belong in what class.

The supervised image resulted in 62.69% accuracy.  This is still not good but it is better than the unsupervised image.  While conducting these it was easy to notice that there are many different shades of green and brown that many ag fields look like when not being used or have weak crop.  These light green brown colors often got confused with bare soil and urban classes.  To get a better result the classification would have to be redone with more meticulous choosing of samples when training the classifier.

Sources:

Cyril Wilson