A deadline to repair the crumbling former eye infirmary has passed without the required work on the Grade II-listed building being done.