As thick black clouds of smoke blew from the factory, city residents complained that there were greasy debris on the streets and cars.
No casualties were reported at this stage, according to Reuters, and regional police said initial tests did not show any serious toxins in the air.
The fire started at 2:40 a.m. Local time, police said, and there may be a strong odor in smoke smoke moving northeast.
Strong odor, thick cloud of smoke , said Paul Brahich, who lives in Rouen, told CNN.
"The fire is still raging and the dense cloud of smoke is still present. A strong odor can be felt outside, causing nausea and pulmonary discomfort," he added.
Rouen resident Amanda Masterson told CNN she was stuck in her home but safe.
"I realized there was a fire when my husband called me and told me that he had heard about it on the radio and that our city was asking us to avoid the house if needed," she said.
"I didn't hear anything, but when I looked out the window above, I saw a huge cloud of dark black smoke. It's really shocking. I'm actually concerned about the effects on health and the environment."
Locals have been sharing videos of leftover oil on the streets, on their cars and on plants in their gardens, Masterson says, while farmers worry about their animals.
CNN's Hilary McGann and Saskia Vandorn contributed to this report.