Your question states prefer open source, but if you’re really open to commercial option, I think the ZingChart JavaScript charting library meets your requirements.

1. Dashboards (easy to create/change): ZingChart uses a CSS-like syntax that is pretty easy to use and edit.

2. Dashboard items should be charts and number boxes: The library allows you to create widgets to display items like you've described. Here is a demo with number boxes -- http://www.zingchart.com/playground/presentation/51b21c1a3c8ae

3 . Warnings on screen: As you can see the in the demo above, rules have been set for the number boxes to display in green for increases and red for decreases. Similar rules can be set for a range of values. (Which addresses number 4).

You could even use multiple rules sets for a values that are close to reaching the threshold. http://zingchart.com/playground/run/5460f51991002 This example shows rules set to place a red marker on data points below the value of 200.

4 . Rule-to-warn: There are a variety of ways to use rules to replicate your desired warning. You can also combine rules with our API and create warnings as well as fire an event which can be used to trigger an email.

Its not mentioned in your question but if real time data is a consideration, ZingChart also provides the ability to transfer data via http or websocket protocols. I’m on the team at ZingChart so if you have any questions about the demo or the features described, please feel free to reach out.