Outages due to weather
/We have three sites down now due to extreme cold and/or solar charging issues, we are loading up to investigate all of them today. We are working to get everything back online as soon as possible.
CLICK HERE to get automated email updates when new posts (including outage information) are added to this page.
We have three sites down now due to extreme cold and/or solar charging issues, we are loading up to investigate all of them today. We are working to get everything back online as soon as possible.
Beginning at 11PM on 2018-12-27 issues with our connectivity in Espanola started happening, originally we thought related to a problem with our connections from REDI Net in that area. That may be true, but shortly after that widespread power outages began in Espanola. We are waiting for more information on what is causing these issues.
The incorrect forwarding problem has been fixed, all 505-852-51XX numbers including the office and support lines should be functional once again.
We are currently sorting out an issue with our phone system with our upstream T1 provider. Incoming calls to 505-852-51XX, including our office line, are not working.
You can still reach us at 844-544-9499 until this is resolved.
We received word from a customer that they got a pop-up telling them to choose a box, and that Black Mesa Wireless was offering some kind of gift.
The way these pop-ups work is that they look up your IP address and see that it is associated with Black Mesa Wireless. This is a feature of the internet IP address registration system (WHOIS/ARIN). They are then pretending to be us. Please close any such ads immediately and do not click them.
We are working with REDI Net to make some changes to our connections with them which will cause brief outages for some customers between 4AM and 5AM on Thursday morning.
The cause of the outage has been narrowed to a failed switch. Service should be restored by 12:00.
We are currently experiencing a tower outage in Alcalde that is affecting clients in Alcalde, Ohkay Owingeh, Chamita, Hernandez, and parts of northern Espanola. We are working towards a solution, but we do not have an ETR at this time.
We will be conducting tower work tomorrow, November 14th, weather/road conditions permitting at Cerro Abajo, affecting the Dixon, Rinconada, and Embudo areas.
Work will include an upgrade to double the speed of the router, and some cleanup work. This tower work may result in 2-3 outages of 1-3 minutes’ duration for customers in these areas as we replace and move equipment.
The result of the work should be improved speeds and reliability of service in the affected areas. Thank you for your understanding as we work to upgrade the network.
We will be moving a major link supplying our Black Mesa tower in the Chamita area to a different location today. This tower has a backup link but it cannot supply the same amount of bandwidth as the main link.
The result of the move will be more than twice the available bandwidth to the tower. It will take place between 08:30 and 12:00 noon on Tuesday, Nov 6th.
Customers connected to this tower may see reduced speeds during this time; however, the backup link can carry the normal amount of morning traffic this tower sees.
The outage in White Rock was found to be caused by a malfunctioning network switch. A spare was swapped in and all systems are again operational in White Rock as of about 11:15am on Saturday Nov 3rd. Thanks for your patience. We have used many of these switches and never had such an issue but these things do happen once in a while.
We will continue to monitor the site closely in case there are further problems.
A piece of equipment in White Rock appears to have failed, we are dispatching someone to investigate the cause, no known ETR.
Due to rain rolling into Espanola earlier than expected, we were not able to complete the tower upgrades as scheduled a few days ago. These will be completed between 09:00 and 12:00 today instead.
We will be upgrading some equipment in Sombrillo tomorrow and a brief outage of up to 30 minutes is expected sometime between 1PM and 2PM. This should substantially improve service in the area.
All services have been restored as of 8:35 AM. Thank you for your patience.
We are currently working on repairing a disconnected line on one of our main backhauls. Clients in the northern section of Espanola, Ohkay Owingeh, San Jose, Chamita, and Alcalde are currently offline. We do not yet have a time estimate for when service will be restored. Please check back for updates.
We just checked in with REDI Net and they are saying another couple hours on the fiber repair.
Here’s a quick summary:
At around 12:30PM today a truck damaged fiber near the China Kitchen in Española, cutting off REDI Net’s access to the outside, and cutting us off from the southern part of our network (White Rock and Pojoaque).
We have another upstream connection, but it is currently maxed out, because we split traffic between REDI Net and the other connection. As of about an hour ago, peak traffic time, the remaining connection became saturated.
Once the fiber is repaired, customers to the south should have service restored. Peak time is almost over at this point, so we anticipate any bandwidth issues caused by this outage should resolve soon.
Thank you for your patience. While this sort of thing is out of our control, we were just meeting with some of the REDI Net staff today to discuss making the southern parts of the network more redundant in case of something like this.
PS. YouTube was down worldwide for a few hours but seems to be working now.
There is currently a REDI-Net outage affecting clients in Pojoaque and causing brief slowdowns of our service. With the exception of those in Pojoaque, service should be online. We do not currently have an ETR, please check back for updates.
Today we completed a significant router upgrade affecting many parts of the Española area, and put in infrastructure to boost throughput further in the Sombrillo area. The Sombrillo upgrades will be completed as soon as we receive some backordered equipment, but we hope some of the new equipment there will boost speeds within the next week for those on the eastern part of Sombrillo road.
With the upgrades today we have found a new bottleneck that we plan to address by purchasing more bandwidth for the affected area effective October 1st at the latest. This will allow us to take full advantage of today’s upgrades.
Thanks for your patience as we continue to grow and upgrade our network. We are committed to removing all bottlenecks on our network as we find them and providing the best possible Internet service in the area.