There are two cases where [comment] button is displayed among those who did not submit the assignment.
The case which a student clicked on an assignment to access the assignment page
The case which an instructor clicked student's [Grades] button to access individual grade evaluation page