Topology inference of non-collaborative wireless network has been extensively investigated for network science and military reconnaissance field. Instead of using linear Granger causality (GC), this letter proposes a novel Difference Neural Granger causality (DNGC) architecture for learning network topology from sampled time series without accessing protocol. With leveraging the hierarchical penalty and differencing approach as an adaptive weight, the proposed DNGC could capture the dynamic and nonlinear connections between neighboring time steps in the collected sequence. Extensive simulations show that the proposed DNGC outperforms the existing GC approaches, especially when the observation time is limited.