Let's examine that idea.
Unless you are reading Genesis in Hebrew, you are reading a Bible which is the result of interpretation by translators. That's true of all English versions of the Bible.
Here's some of the first part of Genesis, in the King James version: (The King James is not perfect, but it is public domain, so it can be used this way; at least the first verse is familiar to most people; and some YECers prefer it).
Genesis 2:1 Thus the heavens and the earth were finished, and all the host of them. 2 And on the seventh day God ended his work which he had made; and he rested on the seventh day from all his work which he had made. 3 And God blessed the seventh day, and sanctified it: because that in it he had rested from all his work which God created and made.
4 These are the generations of the heavens and of the earth when they were created, in the day that the LORD God made the earth and the heavens, 5 And every plant of the field before it was in the earth, and every herb of the field before it grew: for the LORD God had not caused it to rain upon the earth, and there was not a man to till the ground. 6 But there went up a mist from the earth, and watered the whole face of the ground. 7 And the LORD God formed man of the dust of the ground, and breathed into his nostrils the breath of life; and man became a living soul.
24 Therefore shall a man leave his father and his mother, and shall cleave unto his wife: and they shall be one flesh. 25 And they were both naked, the man and his wife, and were not ashamed.
Genesis 4:8 And Cain talked with Abel his brother: and it came to pass, when they were in the field, that Cain rose up against Abel his brother, and slew him.
9 And the LORD said unto Cain, Where is Abel thy brother? And he said, I know not: Am I my brother's keeper? 10 And he said, What hast thou done? the voice of thy brother's blood crieth unto me from the ground. 11 And now art thou cursed from the earth, which hath opened her mouth to receive thy brother's blood from thy hand; 12 When thou tillest the ground, it shall not henceforth yield unto thee her strength; a fugitive and a vagabond shalt thou be in the earth. 13 And Cain said unto the LORD, My punishment is greater than I can bear.
14 Behold, thou hast driven me out this day from the face of the earth; and from thy face shall I be hid; and I shall be a fugitive and a vagabond in the earth; and it shall come to pass, that every one that findeth me shall slay me. 15 And the LORD said unto him, Therefore whosoever slayeth Cain, vengeance shall be taken on him sevenfold. And the LORD set a mark upon Cain, lest any finding him should kill him.
16 And Cain went out from the presence of the LORD, and dwelt in the land of Nod, on the east of Eden. 17 And Cain knew his wife; and she conceived, and bare Enoch*: and he builded a city, and called the name of the city, after the name of his son, Enoch.
25 And Adam knew his wife again; and she bare a son, and called his name Seth: For God, said she, hath appointed me another seed instead of Abel, whom Cain slew.
*Note - according to the Bible, this is not the same Enoch as the one mentioned in Genesis 5:18-24
Genesis 7:11 In the six hundredth year of Noah's life, in the second month, the seventeenth day of the month, the same day were all the fountains of the great deep broken up, and the windows of heaven were opened.
24 And the waters prevailed upon the earth an hundred and fifty days.
Genesis 9:3 Every moving thing that liveth shall be meat for you; even as the green herb have I given you all things.
A response to the statement of the title comes from reading YEC literature. In Six Days of Creation, (Institute for Creation Research, 2013) by Henry Morris, III, a prominent YECer, there is this sentence: "The biblical record indicates that God separated the light from the darkness." Fair enough. That seems to be reading, not interpreting. But the next sentence says this: "This most likely indicates that the earth of Day One was shaped into a sphere (by the Holy Spirit 'energizing' it)." If that isn't interpreting -- going beyond what the Bible actually says -- I don't know what is.
In The New Answers Book 1, (Green Forest, AR: Master Books, 2006. Ken Ham, probably the most prominent of today's YECers, is listed as author on the title page, and as general editor on the cover.) In one chapter, written by Terry Mortenson, presumably with Ham's blessing, we read "The Hebrew words translated 'the fountains of the great deep burst open' (Genesis 7:11) clearly point to tectonic rupturing of the earth's surface in many places for 150 days, resulting in volcanoes, earthquakes, and tsunamis." "Clearly" is an interpretation, and so are the volcanoes, earthquakes and tsunamis.
I don't know the writings of Morris, or Ham (or Ham's assistants), other than superficially. I'm not aware that any of them have ever said "I just read the first part of Genesis. But you interpret it." But some of their followers do make such claims, when, logically, they should not. Interpreting, as Morris and Mortenson have done, can be part of legitimate Biblical scholarship.
As another response to the statement, consider some questions about the text quoted from Genesis:
8) What does "after their kind" mean?
9) What is the "image of God"?
11) Why does Genesis 2:4, in the KJV, say "day"? Didn't the creation take six days, not one?
12) Does Genesis 2:5 mean that plants hadn't appeared completely yet, until Adam did? How can that be reconciled with Genesis 1, which puts plants on the third day?
13) What is the relationship between Genesis 1 and 2:4-25? Is 2:4-25 a re-telling of Genesis 1, or is it an expansion of the sixth day, or is it something else? Genesis 2 leaves out some important things which were given in the description of the six days of creation in Genesis 1. Why?
14) Were there marine bodies of water, as well as fresh water, before the Flood? If so, how did marine organisms survive the increase in fresh water? If not, when did marine and freshwater animals become separated?
15) Where did Cain get his wife? What other people was he afraid of? Did he marry and have offspring before the birth of Seth? If not, why does the mention of Seth's birth come after the story of Cain?
YECs interpret Genesis 1 and 2 to mean that the earth and the living things on it, and other features, were created in six 24 hour days. Perhaps they are right. But there other ways to interpret Genesis, held by Bible-believing scholars who understand Hebrew and the culture of the people of Bible times, that may also be correct. Some important Christians of the distant past held such views.
My conclusion is that anyone saying, about the first part of Genesis, or any other part of the Bible, "I just read it. but you interpret it," should reconsider. We have to interpret it into our own language, and recognize that it was written by people in a different culture. The church of the past often drew conclusions that aren't explicitly stated in scripture, but that most Christians of today consider to be valid, for example on the nature of Christ. May God help us to interpret correctly.
January 27, 2016: I'm adding a link to a relevant conversation on how to interpret Genesis 1, and a link to a relevant interview with John Walton, an influential Old Testament scholar, in Christianity Today. His views on how to interpret Genesis 1 and 2 are important, and influential. Here is one part of a series, examining Walton's views, as set forth in his The Lost World of Genesis One, in the BioLogos blog.