In this thesis, we address the problem of constructing effective hedging strategies against the financial risk of writing a contingent claim in an illiquid financial market. Mathematically, this amounts to study various stochastic optimal control problems with suitable nonlinear dynamics. We introduce a price impact model which accounts for finite market depth, market tightness and finite resilience whose coupled bid- and ask-price dynamics induce convex liquidity costs. We provide existence of an optimal solution to the classical problem of maximizing expected utility from terminal liquidation wealth at some finite planning horizon. In a specific configuration of our model, it turns out that the resulting singular optimal stochastic control problem reduces to a deterministic singular control problem. Rather than studying the associated Hamilton-Jacobi-Bellmann PDE, we exploit convex analytic and calculus of variations techniques which allow us to construct the solution explicitly and to describe analytically the free boundaries of the action- and non-action regions in the underlying state space. In the second part, we relate the optimal singular stochastic control problem of utility-based hedging in our original illiquid market model to a considerably simpler classical linear quadratic stochastic optimal tracking problem of a frictionless hedging strategy with constant coefficients. We solve this problem explicitly for general predictable target hedging strategies. The consideration of general predictable reference processes is made possible by the use of a convex analytic approach along the lines of Pontryagin's maximum principle instead of the more common dynamic programming methods. From a financial point of view, our results allow for an intuitively appealing interpretation and yield sensible hedging strategies in illiquid markets. In the third part, we provide a probabilistic formulation of and solution to a more general class of linear quadratic stochastic tracking problems with stochastic coefficients and stochastic terminal state constraint. Proposing a suitable time consistent approximation of the optimization problem allows us to tackle the final state constraint which induces singular terminal conditions on the underlying backward stochastic differential equations (BSDEs). Our approach also allows us to provide necessary and sufficient conditions under which the constrained stochastic optimization problem admits a finite value. We show that the optimal policy is given by a similar form to the one obtained in the constant coefficient case.