|
Beep Function DYO, Comparison of Studies, Using Global Variables in DYO, Studies or Study Alerts, DYO Memory Usage, MAX Bar Setting, Roll Over, FXCM Update, Data Tab w/IB, Ensign Internet Download Manager, IB Refresh |
March 7, 2007
Sounds with DYOs and Study Alerts
16:31 Ensign Howard: Lets begin by covering the Beep function in the DYO Global Action category.
http://www.ensignsupport.com/email/1540.mhtml
16:31 Ensign Howard: Here is a quick example of adding sound to a DYO.
16:31 Ensign Howard: Line B plays a sound when the GV referenced is True
16:32 Ensign Howard: The Beep sound can be any of 5 different sounds, and the sound is selected by the value in the Number field.
16:32 Ensign Howard: The example shows playing sound #1 when the flag is True and sound #2 when the original flag is False.
(The following information is for XP -- not all other OS are the same.)
16:33 Ensign Howard: The five sounds can be customized on the Windows Control Panel
16:33 Ensign Howard: On the Control Panel you select the program for Sounds and Audio Devices, and on its form select the Sounds tab
16:35 Ensign Howard: The section for Beep #1 will be the sound assigned to the Asterisk selection on the Sounds form.
16:35 Ensign Howard: Beep #2 will play the sound assigned to the Exclamation selection.
16:36 Ensign Howard: Beep #3 will play the sound assigned to the Critical Stop selection.
16:36 Ensign Howard: Beep #4 will play the sound assigned to the Question selection
16:36 Ensign Howard: and all other Number values, such as 0 will play the sound assigned to the Default Beep selection. This association list can be found on the DYO Property form documentation.
16:37 Ensign Howard: When you have a sound played by a DYO, or a Study Alert, it could possibly be designed to play on every tick update of the chart. Sound consumes CPU resources, and could back-log your computer if played too frequently or continuously. Therefore, we imposed a 3 second blackout after playing a sound to block other sounds from playing. We do not queue them up for the very reason it could overwhelm the computer's ability to keep up. If a sound triggers to be played during the blackout period we simply ignore the sound request. So the frequency with which you can play a sound is once every 3 seconds.
16:40 Ensign Howard: This gives the CPU a chance to both play the sound that is to play, and then catch up on processing the data feed, updating the charts and making study calculations.
16:41 Ensign Howard: If you use a Study Alert, you have even more control over sound by being able to specify a path to a sound file to be played. Many will use the sound files in the Windows Media folder Some users will record their own .wav file using a microphone and have a spoken alert. Another sound possibility is to have the text message on the Study Alert copied to the Clipboard so that it can be spoken by another application running such as Speakonia or DeskBot.
(Due to path length restrictions, it is recommended you create a sound folder in ensign directory and copy the wav files to it.)
16:43 Ensign Howard: Any questions about DYO sounds or Study Alert sounds?
16:43 mikeynh: Can the study alert sound be triggered by the logic in a dyo study?
16:45 Ensign Howard: yes, in this manner... have the DYO on the chart ahead of the Study Alert. Then the Study Alert selection can be Study Flag and reference some line in the DYO that is the flag for whether the sound is to play. So the Study Alert is getting its control logic from the DYO, and the Study Alert has the additional path to the wav file or even showing an Alert Message panel.
Comparison of Studies
16:47 Ensign Howard: Now lets move on to another topic and that is the visual interaction of different studies.... some compare well and some do not for the following reason.
http://www.ensignsupport.com/email/1539.mhtml
16:47 Ensign Howard: In this Q&A article I discuss the illogical attempt to compare a MACD with a Simple Moving Average (SMA). The MACD has its own study scale and the SMA uses the chart's price scale. They are unrelated and when the two studies occupy the same sub-window, users often see something they consider a useful signal, but then the signal changes as the data changes and they wonder what is happening and why the signal either disappeared or moved.
16:49 Ensign Howard: Let me show an example using Stochastic and a Moving Average on the chart.
16:51 Ensign Howard: In this example the Moving Average study line is shown in Blue, and the Stochastic %K line is shown in Pink. One might be tempted to treat the pink line crossing the blue line as a study signal to sell show. The crossing relationship is totally arbitrary.
16:51 Ensign Howard: The Stochastic plots on a scale from 0% to 100% and will use the full height of the panel space (chart in this case) in which to draw itself.
16:52 Ensign Howard: The Moving Average is affected by the scale range of the chart, and if the range were to move higher, the moving average would plot lower which the Stochastic line stays in the same position.
16:52 Ensign Howard: Their scales are INDEPENDENT of each other.
16:53 Ensign Howard: Let me illustrate the shift that can occur by changing the scale range of the chart so the MA plots in a different position.
16:54 Ensign Howard: In this example I have moved the range of the chart higher, so the MA line plots lower on the screen, and now the two lines do not cross at all, so the signal disappeared. Now you can see the fickle nature of comparing two studies when they plot on independent scales.
Using Global Variables in DYO, Studies or Study Alerts
16:57 Ensign Howard: OK, on to another topic about using Global Variables in DYOs or with studies or with Study Alerts.
16:57 Ensign Howard: A GV is just a mailbox and it can hold a number. It does not hold any history of what its contents might have been when used earlier in the chart. It only holds the most recent value assigned to it. So if you use a GV assignment on a DYO or on a study, the GV has one value.
16:58 Ensign Howard: let me illustrate.
17:00 Ensign Howard: A study value can be directly assigned or saved to a GV using the Global spinner boxes on the study property form. In this example the 1st Moving Average line value is being saved into GV[5]
17:01 Ensign Howard: If some other chart is trying to read GV[5] to get the moving average line value, it see only the last value put in [5] which is the value for the last bar on the chart. The other chart is not going to see the 1000 different values written by 1000 different bar positions on the original chart.
17:02 Ensign Howard: As a continuation of this topic, let me discuss a Line in a DYO,
17:04 Ensign Howard: Lets say this DYO is on the same chart as the Moving Average that wrote its 1st Average Line value into GV[5]. That would cause the 2 lines in this DYO to be equivalent.
17:04 Ensign Howard: Line A is fetching the line values from the study.
17:04 Ensign Howard: Line B is fetching the same study line values being passed in GV[5]
17:05 Ensign Howard: So both Line A and Line B have identical content. For each bar position on the chart as studies calculate, the program evaluates each study with that bar position.
17:06 Ensign Howard: So when calculating for the 10th bar on the chart, the MA calculates its value, saves it is [5], and then the DYO calculates which will read the MA value and read the GV and have the same value either way .
17:07 Ensign Howard: This data is saved in the 10th position of Line A and the 10th position of Line B because each DYO line is an array which can hold a value at each bar position.
17:07 Ensign Howard: Then when the 11th bar is calculated, the MA calculates and writes a different value go GV[5].
17:08 Ensign Howard: So the GV has no history, but the DYO Lines will have history because the DYO lines are arrays
DYO Memory Usage
17:09 Ensign Howard: But it needs to be discussed because some users have expectation that the GV slot [5] has a history of what its content was at each bar position and such is not the case.
17:10 Ensign Howard: [5] had content at each bar position while the calculation was being done, but after the calculation process, you cannot go back and look at any prior values once put in the GV. In the DYO we do have a history, as in our example, the value for the 10th bar is in the 10th position of the array, and the value for the 11th bar is in the array's 11th position.
17:12 Ensign Howard: You can start to appreciate that the DYO with its 10 Lines is creating in memory 10 arrays to hold values, and each of these arrays is sized to the Max Bars number for how many bars the chart can potentially hold.
17:13 Ensign Howard: If you have a chart with 65,000 bars setting for Max Bars, and you add 10 DYOs to the chart, then the memory usage just sky-rocketed. You have 100 arrays in the 10 DYOS, and each array has 65,000 positions and each position uses 4 bytes of memory.
17:14 Ensign Howard: So lets multiply out the memory consumption in this example: 10 * 10 * 4 * 65,000 = 26 million bytes or 26 Meg
17:15 Ensign Howard: We have a tool in Ensign to give you this memory consumption information automatically. Click menu Help | Memory Report and it will show a little report like the one shown above. The report shows each window in the system, and for charts, the number of bars and the number of study objects on the chart. Each study object has a memory consumption, with DYOs being one of the largest users of memory, and all this is multiplied out to give a memory usage value.
17:17 Ensign Howard: In the example there are 2 windows open, 1 chart and the chat room. The chat room uses 2.2 Meg of memory. The chart with its 5000 bars and 3 study objects uses 0.9 Meg of memory. This is a trivial memory burden, but I am trying to show where you can see for yourself how you are loading up the computer through having charts open and dressed with studies.
17:19 Ensign Howard: Occasionally users ask about their system performing slower and slower, and much of that is happening is they are using more and more memory. They have opened more and more charts, increased Max Bar capacities, and dressed the charts with more studies
17:20 Ensign Howard: We have a web page of suggestions on how to better manage the CPU Burden you are imposing. See this link.
http://www.ensignsoftware.com/help/burden.htm
MAX Bar Settings
17:21 mikeynh: For maximum historical daily data from DTN, what should the max bars setting be in a chart properties window? And must this setting be made individually for each chart? And, can one selectively request more daily bars and fewer say 5-minute bars?
17:22 Ensign Howard: Max Bars setting is a private property for each chart, so you can have ES #F 5-min with 10,000 bars if needed and NQ #F with a different setting of 2,000 bars if needed.
17:23 Ensign Howard: If you think about it, for a Daily chart, there are approximately 250 bars per year, and the maximum futures refresh from DTN might be 13 years, so 13 x 250 would be 3,250 bars. So it does not give you any benefit to try to have 50,000 bars for a daily chart.
17:24 Ensign Howard: In fact, Ensign detects this impossible setting and imposes a smaller maximum on the Max Bars setting for a daily chart to 30,000 bars which is like 120 years of data.
17:25 Ensign Howard: So do yourself a favor, on daily charts, see what the current bar count is on the chart properties, and set you Max Bars level to be a couple years worth (ie 500 more) higher.
17:26 @Buffy2: That is a good way to determine intraday bar count needed also although the volatility lately might justify changing what you were using.
17:27 mikeynh: So for a single symbol, each chart of a different time frame can have a different max bar setting?
17:28 Ensign Howard: Your statement is correct, Mikeynh, each can have a different setting.
17:27 Ensign Howard: That is correct, the ONLY situation I have found that justifies a really big number like 25,000 Max Bars has been our use of the Pesavento Map calculations on a small time frame chart such as the 2-min chart. The MAP needs lots of bars to discover the characteristics it is seeking, and we encourage a minimum of 15,000 bars and find 25,000 to be sufficient.
17:29 Ensign Howard: Now when you open a new chart, it goes to the Setup | Computer form settings to initializes its Max Bar setting to the defaults on this form. We have a default for tick based charts, intra-day charts, and daily charts.
17:30 Ensign Howard: This property form example shows a default of 5000 for both Tick and Daily type charts, and a higher default of 65530 (NOT what I recommend ) for Intra-day. Smaller values are more appropriate for most situations for most users.
17:31 mikeynh: So the minutes setting covers 1-min, 5-min, and so on? What about weeks and months?
17:31 Ensign Howard: Weekly and Monthly share the Daily setting
17:31 @Buffy2: Howard as a reminder is it still correct to say if you have 2 charts same tf the lower max bars rules even if in different workspaces?
17:32 Ensign Howard: Correct buffy, there is really only one property file for the chart though it is member of different workspaces. So you better plan on the file getting clipped to the smaller value. Good point to have as a part of this discussion.
17:33 mikeynh: What do you mean "getting clipped to a smaller value?
17:34 Ensign Howard: Clipped means to be resized so the file is smaller. If the File is to have at most 5000 bars, and it is loaded into memory, and 300 bars are added during the day, only the most recent 5000 will be resaved to the file. The file is not permitted to grow beyond the Max Bars setting. So oldest bars in the file had to be purged to maintain the max bar file size.
17:35 @Buffy2: Some users were trying to have a workspace for trading and keeping more max bars in an other workspace -- only to find all charts had the smaller max bar setting even though different workspaces.
17:35 DarthTrader: ... hmmmm - yes buffy - so that is the equation - I kept loosing data - frustrating for me - what you say makes sense but also makes chart and workspace planning complicated :(
17:36 trendtrader: re: large max bars file setting needed for 2 min charts on Pmaps. might a work around for continuous contract symbols using the #F designations be to use say the ESH7 set to 2000 bars and the ES #F set to 65000 bars?
17:37 Ensign Howard: Good suggestion trendtrader, different symbols create different files and could be different sizes as you suggest.
(You could also use 3000 vol in one and 3001 vol in another workspace.)
Roll Over
17:36 Ensign Howard: ok, on to the topic of Roll-Over of continuation symbols. This topic rolls around every quarter and this week is roll time.
17:38 Ensign Howard: On the eSignal feed they have invented a composite symbol such as ES #F where the #F is their notation for a composite file. eSignal has a roll out schedule where the file currently being updated with the H7 contract will roll to the M7 contract.
17:39 Ensign Howard: The Roll out schedule for the ES by eSignal is the 8 days before the 3rd Friday. Well in March the 3rd Friday is the 16th, and 8 days ahead of that would be the 8th, which is tomorrow. Now all works quite smoothly for eSignal symbols because their refresh database has this composite data from multiple months.
17:40 Ensign Howard: Now, IQFeed also has a composite symbol such as @ES# but the issue gets a bit more complicated for the other data vendors Ensign supports, such as Cybertrader, IB and quote.com.
17:41 Ensign Howard: These data feeds do NOT have a continuation symbols, so Ensign has tried to fill in the need for one by inventing support for a ES #F like is available on the eSignal feed. When you use ES #F as your symbol in Ensign on one of these other feeds, we do not ask for data from IB for ES #F because that is an unknown to them. Instead we RESOLVE what the #F would mean and today it means H7 but later this week it will mean M6. We then substitute in place of the #F a month and year that the feed will accept. Though you are quoting and charting in Ensign ES #F, our interface to the data vendor is with the symbol ES H7 or ES M7.
17:44 Ensign Howard: Now that part works smoothly and is quite transparent to the end user, until we get to roll day and through refresh into the equation. When we ask for refresh for say a week, we can ask for ES H7 or we can ask for ES M7. But we cannot ask for a blend of both
17:45 Ensign Howard: On the eSignal feed we ask for ES #F and get back the blend of both, so eSignal is an easy implementation.
17:45 Ensign Howard: So my suggestion is that on the charts which are looking great today, go to the chart property form and check the Preserve Existing option. That way after the roll day, even if you ask for a big refresh that would span back in time to overlay the good data you have now built from the H7 data, the refresh will not be able to replace and thereby corrupt your H7 data. Later this week we will be following M7 data and requesting refresh from M7
17:47 Ensign Howard: Check the Preserve Existing will keep you bars built from the H7 data instead of letting them be overwritten by bars returned historically for M7
17:48 Ensign Howard: I hope I have clarified what is happening and why, and the best way to prevent overwriting your continuation file content with M7 bars.
17:48 Ensign Howard: any questions?
17:48 mikeynh: So, as a general rule, Preserve Existing should always be checked in all cases?
17:48 Ensign Howard: I typically have mine checked. IF and when I see a bar or several bars that needs to be replaced by the refresh, I manually delete them using the erase tool. The refresh can insert bars that are missing regardless of the Preserve Existing check box state.
17:50 mikeynh: So, when trading tomorrow, we should be trading the June contract because the freshest bars will be returned (by IB) based on a request for the M7 contract?
17:50 Ensign Howard: That is correct Mikeynh, unless you have a different roll schedule in place for the symbol. The roll schedule is unique for each symbol, and the most common symbol of ES uses 3rd Friday - 8 days. This schedule you see on the chart properties (CTRL+P keys) on the Symbol tab, lower right corner.
17:52 Ensign Howard: In the example, the Months box shows this symbols trades in the months of MHUZ, So when we get to March (H month), the roll day in that month is on a Thursday 8 days ahead of the 3rd Friday. The sequence is HMUZ as shown.
FXCM Update
17:50 Ensign Howard: Ok, now for my final training topic.... the forex feed from FXCM.
17:55 Ensign Howard: Ensign developed in interface and servers to handle a free feed from FXCM for their forex currencies. This was a monumental project in 2006 and we released the feed for use by users in early January of 2007. Things have gone very well with few glitches over the past 2 months....
17:56 Ensign Howard: However, we have had a few days with high volume or high activity and the feed from fxcm lagged behind, which we considered to be of real concern and unacceptable. We have talked with FXCM and got them to acknowledge the problem can happen. The queuing of the data is happening on their server side, and thus is not in the control of Ensign to change. They did say they have a way to access quotes directly from their order desk before these quote go into the queue for the data feed.
17:58 Ensign Howard: Ahhh.... that opened up a solution to get the feed from the source as it happens before it goes into the feed queue for distribution on the feed we were processing. So we have proceeded with haste to tap into this order desk source, and have had the new feed working well for the past 3 days.
18:00 Ensign Howard: We are continuing to make changes to our implementation to make it more efficient, and will continue to test this feed ourselves with the expectation that if all continues to go well, we will make the change this coming Monday to replace the current forex feed from FXCM with a data stream directly from their order desk.
18:01 Ensign Howard: I had hoped to see a period of high activity so I could compare the two feed side by side, and see lag in the existing and no lag in the new. However, there has not been a period of such high activity to cause the existing to lag this week. So, I continue to test and compare the two feeds. Anyway, that is an update on what we are doing and why and where we are headed with the free forex feed from FXCM.
18:03 Ensign Howard: We intend to pick up 2 advantages by the new implementation and the primary advantage is to eliminate any lag in periods of high activity and the 2nd unexpected advantage is we can get data on 12 more currency pairs that are not released on the regular feed. That will give our implementation 54 currency pairings on our forex feed. So I guess cleverness can pay off in additional ways
IB Refresh
18:04 mikeynh: With IB as my broker, should my charts be set to use IB as the feed or DTN? Or, more generally, why and how should I use DTN, if at all?
http://www.ensignsupport.com/email/1526.mhtml
18:05 Ensign Howard: IB refresh has become more problematic since the first of the year and the cause is IB has imposed a pacing limitation for how much refresh you can get in a 10 minute period. Read the above document about Pacing Violations. If you ask for too much refresh then IB will block further requests by ignoring you. Details are in the article.
18:07 Ensign Howard: Now DTN as a 2nd refresh resource is a blessing because it give you an alternative and can help you manage the IB limit better. You do not have a limit on the DTN refresh frequency. So if you have a chart that you can refresh from DTN, do so, and that will off load counting against the IB limit. DTN refresh advantage is they have more data so you can refresh further back in time, and the refresh is received much quicker. Disadvantage of DTN is that they impose the delay requirement so if the requirement is 10-minutes, the refresh will be through 10 minutes ago.
18:10 mikeynh: So after backfilling the data, it makes sense to switch back to using IB because there will be no delay?
18:10 Ensign Howard: The only advantage of IB is that their refresh will be through the current tick, instead of through the delay requirement. The disadvantage, is they count the refresh requests, and each day for intra-day charts is a request, and each 33 minute on a tick based chart is a request. As explained in the article, try to minimize the IB refreshing to use just the 33 min selection for the tick based refreshing and 1 day for the intra-day charts. In general, I discourage refreshing tick based charts when you have been on-line and already collected and built the chart bars.
18:13 Ensign Howard: Neither DTN nor IB tick refresh is going to exactly match the tick you collected in real-time form IB, and so any refresh of ticks is going to cause some change in your tick based charts. This change is upsetting to some users because their study signals changed.
18:13 Ensign Howard: Sorry, we cannot control that. The reason for the difference is this.....
18:14 Ensign Howard: IB feed does not send every tick because they are a sampled feed......
18:15 Ensign Howard: The DTN refresh is sending every tick, and so the DTN refresh will cause more bars to be build for Constant Tick bars than you collected in real-time.
18:16 Ensign Howard: And the IB refresh does not refresh ticks,... the smallest refresh they have is a 1 second bar refresh, and we use that 1-second bar and try to reconstitute what multiple ticks (if any) might have made up the 1 second time period. Most of the time the 1 second refresh bar has 4 identical prices so we can assume it has 1 tick and all the volume belongs on that tick. If we have an opening price that differs from the closing price, we assume there are 2 ticks and split the volume for the record between the open tick (first) and the closing tick (2nd).
18:17 mikeynh: So, bottomline, when trading, which feed should I use?
18:18 Ensign Howard: if you use IB feed, trade with it... it is a great feed. We have more IB customers than any other feed. I am just pointing out the reason for discouraging refreshing of tick based charts when you have been on-line and already built a nice chart. The chart is not going to be improved upon by trying to refresh it. Some bars will change in the refresh from either source, so keep what you have and be happy
18:20 Ensign Howard: If this live feed versus refresh is irritating to you, then you do have alternatives of paying more money for a data feed from IQFeed or from eSignal or from Cybertrader. Their refreshes match their live feeds
18:21 mikeynh: Not irritating, it's just that I would like to accumulate the historical data, being a new user, and then be happy using IB.
18:22 Ensign Howard: Go ahead and accumulate... the past is the past and not something you can trade after the fact anyway
Ensign Internet Download Manager
18:23 FuturesTrader: Is there any disadvantage to not keeping futures contract months up to date in the Download Manager if the Quotes list is up to date and charts are set to Auto Refresh? Using Ensign Internet.
18:24 Ensign Howard: Good question.... when originally developed, the Download Manager list was the only way to get the chart data . Through program evolution, we implemented asking for the chart refresh on demand by just opening the chart. So this 2nd method proves to be easier and more intuitive.
18:25 Ensign Howard: And this has kind of obsoleted the need for the Download list, unless you are harvesting data for a long list and want to do so as an event you run each evening. So the answer your question, you can basically abandon the Download list in light of the way you are using the program and workspaces to open your charts. Having the Auto Refresh option enabled on the chart properties does an auto refresh request to get the chart data set as needed.
IB Data Tab
18:28 bl: On the Chart properties - data tab - there is a "select" box. It has a choice of "ER2 H7" or "Continuation" (in my case) with an IB data feed What is the difference in boxes?
18:29 Ensign Howard: When the selection is on IB for the source, the IB feed is used and they only have the symbol for a specific month.
18:30 Ensign Howard: When the selection is on DTN they have two possible symbols we could use and the Select box choose which is used.
18:30 Ensign Howard: The ER2 H7 would be the data set for the specific month
18:30 Ensign Howard: The Continuation would be the ER2# symbol which is DTN's continuation symbol
18:31 Ensign Howard: HOWEVER, DTN uses a different roll schedule calendar than is used by eSignal and that causes frustration by users on different feeds who all want to be the same. So Ensign still has our own ER2 #F continuation symbol controlled by the roll schedule calendar on the chart properties. This will permit you to roll when you want to in spite of when DTN elects to roll their ER2# symbol
18:32 Ensign Howard: So ER2# is composite data, part Z6, part H7 and part M7 to begin later this month. Tomorrow the selection choice will show ER2 M7 and continuation and you probably will need to be on the ER2 M7 selection to get refresh for M7 for your charts. As discussed earlier in class, have the Preserve Existing Bar option checked so the chart which is good today does not let the refresh replace any bars.
18:35 Ensign Howard: Any other questions... if not, lets call it a day... we have been at it for 2 hours and this has been another good class.
18:35 @Buffy2: yes it has. Excellent. Thank you very much.
18:36 bl: Thank you Howard
18:36 Rice: thanks H
18:36 Ensign Howard: you are all welcome....
18:37 FuturesTrader: Thank you so much for your awesome help and training. Your help DYO last week helped me figure out how to do some signals that are just sooo great!!!
18:38 Ensign Howard: thx FuturesTrader,..... it is rewarding to us when users benefit from our efforts to develop a good product and offer training on its use.
18:38 FuturesTrader: <=== Very happy and pleased customer :)
18:38 Ensign Howard: good night