Can there be buildup on walls from a prior smoker that pollutes the indoor air?
Unfortunately, there are many steps in this process where shortcuts can be taken, and property management companies and their employees are under great temptation to take every short cut that will save them money. Without knowing your specific allergies and testing the apartment, it would be impossible to pinpoint the cause of your increased irritation and required inhaler use. But, your observations and smells are valid, and the causal link you make to your inhaler use is reasonable.
In other words, you make a good argument, and your property manager should take your concern seriously. If you've been a good tenant in the complex for 3 years, your management company should take your word that there is a problem in the apartment unit and move you to another one. If they don't agree to move you to a comparable unit, put your complaint in writing with as much detail as possible. Tell them what you think the problem is, and ask to be a moved to a unit where there has been no smoking, or not much smoking and a thorough proper cleaning. Keep track of all correspondence, and make notes after each conversation. If that doesn't get you moved to a better unit, contact the health department, the housing authority and potentially an attorney (perhaps with a legal aid service if you can't afford a real estate attorney).
Property owners have a responsibility to provide clean, safe housing and to make reasonable accomodations for your needs. The laws in your state govern the specifics of landlord-tenant law, so if talking to your property manager doesn't resolve the issue, talk to a knowledgable professional about next steps.