Visual messages are the dominant form of communication today, and the meaning of literacy has been broadened to include the language of images and other modes. Yet, in most early childhood education settings in Taiwan, the emphasis of literacy education is still on the reading and writing of words. This study aimed to fill this gap by investigating how Taiwanese young children observe and interpret images in wordless picture books, and compared the findings to studies done in the West. Eight 6-year-old children read three wordless picture books—Wolf, Rainstorm and The Grey Lady and the Strawberry Snatcher—with the investigator individually. They read the pictures and told the stories in their own words, followed by interviews which asked about their interpretation of the pictorial elements in the books. The children also drew pictures of the things that impressed them the most from the books, and this activity was followed by interviews with the researcher about the drawings. The findings showed that young children’s reading experiences played an important role in how they read the peritextual features of picture books. Young children observed more than ten different kinds of visual elements in the wordless books. Both the stories they told and their drawings showed how they perceived the visual elements and pictorial compositions in the books. Moreover, the children made sense of the wordless books by constructing a coherent plot, and they had diverse interpretations towards certain elements in the books.