Abstract
One of the most dangerous cybersecurity threats is control hijacking attacks, which hijack the control of a victim application, and execute arbitrary system calls assuming the identity of the victim program’s effective user. System call monitoring has been touted as an effective defense against control hijacking attacks because it could prevent remote attackers from inflicting damage upon a victim system even if they can successfully compromise certain applications running on the system. However, the Achilles’ heel of the system call monitoring approach is the construction of accurate system call behavior model that minimizes false positives and negatives. This paper describes the design, implementation, and evaluation of a Program semantics-Aware Intrusion Detection system called Paid, which automatically derives an application-specific system call behavior model from the application’s source code, and checks the application’s run-time system call pattern against this model to thwart any control hijacking attacks. The per-application behavior model is in the form of the sites and ordering of system calls made in the application, as well as its partial control flow. Experiments on a fully working Paid prototype show that Paid can indeed stop attacks that exploit non-standard security holes, such as format string attacks that modify function pointers, and that the run-time latency and throughput penalty of Paid are under 11.66% and 10.44%, respectively, for a set of production-mode network server applications including Apache, Sendmail, Ftp daemon, etc.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Acharya, A., Mandar, R.: Mapbox: Using parameterized behavior classes to confine untrusted applications. In: Proceedings of the Tenth USENIX Security Symposium (2000)
Alexandrov, A., Kmiec, P., Schauser, K.: Consh: A confined execution environment for internet computations. In: USENIX Ann. Technical Conf. (1999)
Balfanz, D., Simon, D.R.: Windowbox: a simple security model for the connected desktop. In: Proceedings of the 4th USENIX Windows Systems Symposium, pp. 37–48 (2000)
CERT Corrdingation Center. Cert summary cs-2003-01, http://www.cert.org/summaries/ (2003)
cker Chiueh, T., Hsu, F.-H.: Rad: A compiler time solution to buffer overflow attacks. In: Proceedings of International Conference on Distributed Computing Systems (ICDCS), Phoenix, Arizona (April 2001)
Cowan, C., Pu, C., Maier, D., Hinton, H., Walpole, J., Bakke, P., Beattie, S., Grier, A., Wagle, P., Zhang, Q.: Stackguard: Automatic adaptive detection and prevention of buffer-overflow attacks. In: Proceedings of the Seventh USENIX Security Symposium, San Antonio, Texas, January 1998, pp. 63–78 (1998)
Etho, H.: Gcc extension for protecting applications from stack-smashing attacks, http://www.trl.ibm.com/projects/security/ssp/
Feng, H.H., Kolesnikov, O.M., Fogla, P., Lee, W., Gong, W.: Anomaly detection using call stack information. In: Proceedings of the IEEE Symposium on Security and Privacy, Berkeley, CA, May 2003, pp. 62–76. IEEE Press, Los Alamitos (2003)
Giffin, J.T., Jha, S., Miller, B.P.: Detecting manipulated remote call streams. In: USENIX Security Symposium (August 2002)
Giffin, J.T., Jha, S., Miller, B.P.: Efficient context-sensitive intrusion detection. In: 11th Annual Network and Distributed System Security Symposium (February 2004)
Goldberg, I., Wagner, D., Thomas, R., Brewer, E.A.: A secure environment for untrusted helper applications. In: Proceedings of the 6th Usenix Security Symposium, San Jose, CA, USA (1996)
Hastings, R., Joyce, B.: Purify: Fast detection of memory leaks and access errors. In: Proceedings of the Winter USENIX Conference, pp. 125–136 (1992)
Hofmeyr, S., Forrest, S., Somayaji, A.: Intrusion detection using sequences of system calls. Journal of Computer Security 6(3) (1998)
Kiriansky, V., Bruening, D., Amarasinghe, S.: Secure execution via program shepherding. In: 11th USENIX Security Symposium (August 2002)
Nguyen, N., Reiher, P., Kuenning, G.H.: Detecting insider threats by monitoring system call activity. In: IEEE Information Assurance Workshop, United States Military Academy West Point, New York (June 2003)
Prasad, M., cker Chiueh, T.: A binary rewriting approach to stack-based buffer overflow attacks. In: Proceedings of 2003 USENIX Conference (June 2003)
Prevelakis, V., Spinellis, D.: Sandboxing applications. In: Proceedings of the FREENIX Track: 2001 USENIX Annual Technical Conference, Berkeley, CA, June 2001, pp. 119–126. USENIX Association (2001)
Sekar, R., Bendre, M., Bollineni, P., Dhurjati, D.: A fast automaton-based method for detecting anomalous program behaviors. IEEE Symposium on Security and Privacy, 144–155 (2001)
Solar Designer. Non-executable user stack, http://www.false.com/security/linux-stack/
TESO Security. x86/linux wu ftpd remote root exploit, http://packetstormsecurity.nl/0205-exploits/7350wurm.c
Vendicator. Stackshield: A “stack smashing” technique protection tool for linux, http://www.angelfire.com/sk/stackshield/
Wagner, D., Dean, D.: Intrusion detection via static analysis. In: Proceedings of the IEEE Symposium on Security and Privacy, Oakland, CA, May 2001, pp. 156–169. IEEE Press, Los Alamitos (2001)
Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security (November 2002)
Warrender, C., Forrest, S., Pearlmutter, B.: Detecting intrusions using system calls: Alternative data models (May 1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lam, L.C., Chiueh, Tc. (2004). Automatic Extraction of Accurate Application-Specific Sandboxing Policy. In: Jonsson, E., Valdes, A., Almgren, M. (eds) Recent Advances in Intrusion Detection. RAID 2004. Lecture Notes in Computer Science, vol 3224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30143-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-30143-1_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23123-3
Online ISBN: 978-3-540-30143-1
eBook Packages: Springer Book Archive