They must have notochord at every point of their life
They have a post anal tail
They have hollow dorsal nerve cord
They have a ventral heart