Joseph provided bread for Israel in Egypt. In fact, Joseph provided bread for most of the surrounding world in Egypt.
It would not be hard for Israel to slip into seeing the systems and culture of Egypt as the source of Yahweh’s provision of this bread (which is partially true), and from there, it is only a short step to mistaking Egypt as the source of bread rather than Yahweh.
Thus, when Yahweh orders Israel to leave the leaven of Egypt behind, He is insisting that He is the One who provides bread for Israel. God can rain bread and meat out of heaven and cause water to gush out of rocks if He wants. And sometimes He uses obvious means (like a prosperous nation) to provide for His people. But either way it is Yahweh who provides bread for Israel, and in so far as Israel comes to trust in Egypt for bread, they are idolaters.