Modular forms are characterized by certain transformation laws. In general, these properties are not preserved by (holomorphic) differentiation. I will discuss several ways to overcome this trouble: Analytically one can use non-holomorphic differential operators (‘‘Maaß-Shimura operators’’) or use bilinear holomorphic differential operators (‘‘Rankin brackets’’). More arithmetically, one can show that derivatives of modular forms are still congruent to modular forms (‘‘Ramanujan’s $\theta$-operators’’). I will describe the connection between the analytic and the arithmetic aspects with focus on some recent results on $\theta$-operators.