We propose and demonstrate a method for improving the sensitivity of fiber Bragg grating (FBG) based accelerometers. The method employs a short FBG inscribed in single mode fiber (SMF) with cladding substantially thinner than 125 μm. The resultant stress concentration at the FBG significantly improved the axial strain response compared with 125-μm-diameter fiber. A 50-μm-diameter FBG fixed on a cantilever beam was used to experimentally demonstrate this sensitivity enhancement. The accelerometer has a working frequency bandwidth from 0.5 Hz to 30 Hz with a sensitivity of 2150±30 pm/g, amounting to 5-fold sensitivity enhancement compared to 125-μm-diameter fiber.