Model-based Bayesian experimental design aims to enhance learning about model parameters of interest by identifying experimental configurations that maximize the informativeness of data. This informativeness is quantified through metrics such as Shannon Expected Information Gain (EIG), which measures the reduction in uncertainty about a parameter given experimental observations. For non-linear models, EIG provides a robust criterion grounded in information theory, though its evaluation through classical methods is computationally expensive and susceptible to challenges such as numerical underflow. In the context of parametric differential equations, optimizing the design of experiments is crucial for efficiently gaining valuable insights while minimizing computational cost. This is particularly vital in high-dimensional parameter spaces, where solving differential equations repeatedly for varying parameter values is highly expensive. Adaptive approaches to Bayesian experimental design, utilizing effective sampling and dynamic error control strategies, offer a pathway to significantly reduce computational overhead while ensuring accuracy. We present an adaptive stochastic optimization approach to this challenge by employing Stochastic Gradient Descent (SGD) with adaptive sample sizes for EIG estimation. Central to this framework is the Double-Loop Monte Carlo (DLMC) estimator of the gradients, designed to minimize average computational cost, while providing theoretical guarantees on convergence and error bounds under strong-convexity and smoothness assumptions. The method further incorporates a bootstrap-based adaptive stopping criterion and dynamically adjusts discretization parameters. Numerical experiments, including applications like optimal sensor placement, illustrate the method's effectiveness. These results highlight the applicability of adaptive Bayesian experimental design for computationally intensive EIG-based parameter learning in parametric differential equations.