Would it be possible to add a feature (if not already existing, sorry I'm new to this engine) that would allow putting in maybe a prerendered video in places? And I would also like to know if it would be possible to have text dialogue on entities when you walk up to them or maybe interact with them

Read a manual first before asking such questions. Everything you asked is possible.

Prerendered video: https://pixelwolf.net/efpse/wiki/index.php?title=System_Resources

Text dialogue on entities when you interact with them? Do this

1. go to the Scripts folder in Easy FPS Maker
2. Make new txt file
3. Name it based on the entity you want to interact with (e.g. Giant Rat) and save it as (entity name).script for example GiantRat.script
4. paste the following:

text "I'm a rat, yay"

5. In the game, approach Giant Rat and press E. He will say "I'm a rat, yay"